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IV.  Introduction 


A  SUBJECT  OF  THE  RESEARCH  AND  PROBLEM  STATEMENT 

Breast  cancer  is  the  most  common  cancer  among  women,  and  the  second  leading  cause  of 
death  among  women.  In  the  State  of  Maryland,  where  the  major  effort  of  the 
telecommunications  Breast  Health  Information  Project  took  place,  the  incidence  of  breast 
cancer  ranks  the  seventeenth  worst  in  the  nation  (1995  Cancer  Facts  &  Figures). 
Unfortunately,  the  overall  statistics  mask  the  real  disparity  of  breast  cancer  incidence 
between  White  and  African  American  females. 

During  the  past  20  years,  the  prospects  of  survival  for  young  white  women  diagnosed 
with  breast  cancer  have  improved,  while  the  prospects  for  black  women,  especially  older 
ones,  are  increasingly  grim.  African-Americans  with  breast  cancer  face  more  than  twice 
the  risk  of  dying  compared  with  WTiite  Americans,  primarily  because  they  are  diagnosed 
at  more  advanced  stages  of  disease. 

In  addition  to  this  alarming  statistic,  the  American  Cancer  Society’s  1989  “Report  to  the 
Nation  on  Cancer”  states  that  cancer  education  and  outreach  efforts  have  not  penetrated 
poor  communities  or  tapped  into  the  right  information  networks. 

The  Breast  Health  Information  Project  (BHIP)  represents  a  multi-year  research  effort  that 
assessed  the  effectiveness  of  automated  messaging  information  systems  in  reaching 
minority  and  underserved  women,  and  encouraging  them  to  take  proactive,  preventive 
care  steps  against  breast  cancer.  The  research  was  conducted  by  the  TeleSonic  Division 
of  Associated  Enterprises,  Inc.,  an  Annapolis,  Maryland-based  telecommunications 
company. 


B.  BACKGROUND  OF  PREVIOUS  WORK 

The  research  conducted  builds  upon  TeleSonic’ s  earlier  telecommrmications  research  in 
the  area  of  cancer  health  information  systems. 

In  a  previous  telephone  information  system  development  project  conducted  by  TeleSonic 
calling  volume  exceeded  400  percent  of  goal.  This  project  was  conducted  for  the  Office 
of  the  Secretary,  Office  of  Family  Policy  and  Support,  Department  of  Defense,  in  which 
calling  volume  was  measured  to  a  job  bank  telephone  information  system  among  military 
families.  This  joint  service  project  included  Army  Community  Services,  Navy  Family 
Service  Centers,  the  Marines,  and  the  Air  Force  Family  Support  Center.  It  was  more 
successful  than  anyone  could  have  ever  predicted.  The  project  clearly  showed  that  lower 
income  and  hard-to-reach  individuals  with  generally  lower  educational  training 
backgrounds  will  seek  out  information  that  is  provided  in  a  convenient,  readily  accessible 
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telephone  format,  when  made  available.  The  implications  for  reaching  the  same  audience 
of  individuals  with  pertinent  breast  cancer  information  have  future  benefit  potential 
beyond  the  scope  of  this  project. 


C.  PURPOSE  OF  WORK 

The  Breast  Health  Information  Project  (BHIP)  was  a  three-year  research  effort  conducted 
by  TeleSonic,  with  funding  from  the  U.S.  Department  of  the  Army,  Grant  #DAMD  17-94- 
J-4282A. 

The  objective  of  TeleSonic's  three-year  study  was  to  determine  if  automated  telephone 
information  systems  are  an  easy  and  effective  way  to  communicate  information  about 
breast  cancer  prevention  to  informationally  hard-to-reach  populations.  For  this  research, 
the  informationally-hard-to-reach  population  was  defined  as  African  Americans,  and  both 
African  American  and  White  low-to-moderate  income  women.  TeleSonic  examined  user 
preferences  for  an  automated  telephone  information  system  versus  a  live  counselor 
referral  service  to  determine  if  any  statistically  significant  preferences  exist  among  racial 
and  income  groups. 

The  research  conducted  builds  upon  the  TeleSonic’s  earlier  telecommunications  research 
in  the  area  of  cancer  health  information  systems.  For  the  BHIP  research,  TeleSonic 
developed  an  automated  information  system  to  be  used  in  communicating  information 
about  breast  health  and  breast  cancer.  Additionally,  the  company  established  a  testing 
approach  to  evaluate  the  effectiveness  of  an  automated  telephone  information  system  in 
increasing  access  to  information,  as  compared  to  a  live  counselor  referral  service.  This 
increased  access  to  information  is  believed  to  be  crucial  in  encouraging  greater  numbers 
of  minority  and  underserved  females  to  seek  early  diagnosis  and  appropriate  treatment  of 
breast  cancer. 


D.  SCOPE  OF  THE  RESEARCH  AND  METHODS  OF  APPROACH 
Approach 

TeleSonic  has  taken  a  new  and  innovative  approach  to  fostering  behavior  changes  among 
health  care  recipients.  Rather  than  look  at  the  issue  strictly  as  a  health  problem, 
TeleSonic  has  approached  it  as  a  marketing  problem.  The  result  is  a  unique  and  cutting- 
edge  technology  solution  that  cost-effectively  pinpoints  target  populations,  and  provides 
them  with  preventive  health  care  information  in  a  convenient,  private,  and  non¬ 
threatening  way. 

TeleSonic's  approach  combines  four  major  components;  (1)  A  geographic  mapping 
process  for  pinpointing  target  groups  according  to  defined  census  criteria;  (2)  A  targeted 
direct  mail  campaign  for  promoting  a  free  preventive  health  information  service;  (3)  An 
automated  telephone  information  system  that  provides  free  24-hour  automated  health 
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information;  and  (4)  An  ad  hoc  report  generator  that  tracks  direct  mail  response  rates 
by  census  criteria  and  measures  respondent  behavior  levels. 

After  defining  target  populations  and  identifying  them  geographically  through  a  mapping 
process,  TeleSonic  used  a  direct  mail  campaign  to  promote  both  an  automated  telephone 
information  system  and  a  live  counselor  referral  system.  The  company  then  compared 
the  response  rates  by  census  criteria  and  measured  caller  behaviors  using  sophisticated 
data  analysis  techniques. 

As  part  of  the  research,  TeleSonic  designed  an  automated  telephone  information  system 
that  provides  detailed  information  about  breast  cancer.  The  system  offers  over  20 
preventive  care  audio  messages  on  topics  including  breast  cancer  risk  factors,  ways  to 
lower  breast  cancer  risk,  how  to  do  a  breast  self-examination,  when  to  get  a 
mammogram,  and  where  to  get  more  information.  The  system  also  offers  callers  the 
option  to  get  additional  information  by  mail  or  fax,  and/or  to  transfer  to  a  live  counselor. 

In  addition,  TeleSonic  developed  a  highly  specialized  geographic  targeting  technique  that 
segments  zip  code  areas  into  census  block  groups,  pinpoints  geographic  concentration 
areas  of  specific  populations  by  demographic  and  psychographic  criteria.  The  project 
cost-effectively  reached  these  populations  with  preventive  health  care  information. 
TeleSonic's  approach  integrated  geographic  mapping  and  audiotex  technology  as  a 
proven,  effective  means  of  reaching  "informationally  hard-to-reach"  populations  with 
health  information  and  fostering  them  to  take  preventive  health  care  actions. 


Goals  of  the  Research 

TeleSonic's  hypothesis  stated  that  using  a  multimedia  automated  telephone  information 
system  is  an  effective  means  of  reaching  "informationally  hard-to-reach"  populations,  and 
getting  them  to  take  proactive,  preventive  care  steps  against  breast  cancer. 

TeleSonic  identified  several  key  evaluation  questions  for  measuring  the  success  of  its 
hypothesis.  These  evaluation  questions  included:  (1)  Determining  if  the  use  of 
telecommunications  increased  the  number  of  females  seeking  information  about  breast 
cancer,  and  if  so,  determining  if  the  rate  of  increase  varied  by  race  or  social-economic 
status  (SES).  (2)  Determining  whether  callers  had  a  preference  for  an  automated 
telephone  information  system  versus  a  live  counselor  referral  service  for  obtaining 
information  about  breast  cancer.  (3)  Determining  if  the  increased  access  to  breast  cancer 
information  resulted  in  increased  proactive  actions  such  as  screening  and  physician 
appointments,  and  if  so,  whether  subsequent  actions  varied  by  race  and/or  SES? 

To  answer  the  research  questions,  TeleSonic  measured  three  different  levels  of  behavior 
by  race  and  income:  (1)  Response  to  the  direct  mail  campaign  (by  calling  the  automated 
telephone  information  system  or  the  live  counselor  referral  service);  (2)  Information 
gathering  actions  taken  while  using  the  automated  telephone  information  system;  and  (3) 
Subsequent  preventive  care  actions  taken  as  a  result  of  using  the  automated  information 


system.  These  three  different  levels  of  behavior  were  designated  as  behavior  levels  one, 
two,  and  three. 

Behavior  Level  One  measured  caller  response  rates  to  a  targeted,  controlled  direct  mail 
campaign.  For  this  measurement,  call  counts  into  the  telephone  system  were  tracked  and 
compared  to  the  total  number  of  pieces  mailed.  Because  this  was  the  most  basic  of  the 
three  behavior  levels,  TeleSonic  expected  the  greatest  response  in  this  category. 

Behavior  Level  Two  measured  what  actions  callers  took  when  they  called  the  automated 
telephone  system.  Actions  in  this  behavior  category  indicated  how  actively  callers 
utilized  the  breast  cancer  prevention  information  presented.  For  this  behavior  level, 
TeleSonic  measured  caller  actions  such  as  time-on-line,  number  of  messages  listened  to, 
transfers  to  a  live  counselor,  request  for  information  by  mail  or  fax,  completion  of  a 
"personal  profile"  assessment",  and  participation  in  an  optional  survey. 

Behavior  Level  Three,  the  most  difficult  category  to  measure,  examined  subsequent 
actions  of  callers  to  determine  if  they  took  any  new  preventive  and/or  proactive  breast 
cancer  prevention  actions  as  a  result  of  using  the  telephone  information  system.  Behavior 
Level  Three  actions  include  such  proactive  steps  as  practicing  breast  self-examination, 
scheduling  an  appointment  with  a  medical  practitioner,  or  scheduling  a  mammogram. 

TeleSonic  also  identified  anticipated  outcomes  for  the  three-year  research  period,  as 
follows: 

(1)  The  successful  development  and  operation  of  an  innovative  multimedia  breast  cancer 
telephone  information  system 

(2)  An  increase  in  females,  taking  proactive  breast  cancer  prevention  and  diagnosis  steps 

(3)  The  collection  of  previously  unavailable  data  to  measure  the  effectiveness  of  the 
system  in  reaching  minority  and  underserved  populations 

(4)  The  enhancement  of  currently  available  breast  cancer  information  for  use  by 
physicians,  patients,  and  concerned  family  members. 

Identified  Tasks 

To  answer  the  research  questions,  TeleSonic  developed  nine  major  tasks,  as  follows: 

Task  1  :Determine  content  of  the  information  system 
Task  2:  Research  technical  system  design  issues 

Task  3:  Research  issues/topics  and  produce  messages  identified  for  the  information 
system’s  audiotex/faxiotex  library 
Task  4:  Acquire  and  set  up  the  demonstration  system  model 
Task  5:  Establish  system  and  data  retrieval  routines 

Task  6:  Develop  and  implement  the  outreach  and  promotional  campaign  plan 


4 


Task  7:  Conduct  at  least  two  comparable  tests  of  the  system 
Task  8:  Research,  define,  and  identify  appropriate  future  modes  of  sustaining  the 
information  beyond  the  test  phase 
Task  9:  Document  and  report  results 

The  information  presented  in  Appendix  A,  Project  Work  Plan,  highlights  these  tasks. 


Testing  Strategy 

To  test  its  hypothesis,  TeleSonic  identified  four  main  target  groups  by  race  and  income 
level:  a  White  Low-Moderate  income  group  (WLM),  a  White  Middle-Upper  income 
group  (WMU),  an  African  American  Low-Moderate  income  group  (AALM),  and  an 
African  American  Middle-Upper  group  (AAMU).  This  structure  allowed  TeleSonic  to 
measure  response  rates  to  a  direct  mail  campaign  by  both  race  and  income. 

The  overall  research  design  encompassed  one  pre-test  and  four  test  efforts,  as  follows: 

•  A  Pre-Test,  conducted  in  Baltimore,  Maryland,  prior  to  initiating  Test  One.  This 
pre-test  examined  the  response  patterns  of  callers,  using  a  small  sampling  from  the 
targeted  households.  The  pretest  was  designed  to  provide  information  to  help  determine 
the  number  of  mailings  for  Test  One  and  subsequent  tests.  In  addition,  the  pretest  was 
designed  to  obtain  any  information  regarding  the  subjects’  reactions  to  the  content  and 
the  overall  system.  This  information  was  used  to  make  adjustments  to  the  system  before 
Test  One. 

•  Test  One,  conducted  in  Baltimore,  Maryland.  This  test  was  a  comparative 
assessment  of  caller  preferences  in  four  target  groups  for  an  automated  telephone 
information  system  versus  a  live  counselor  referral  service.  Test  results  were 
statistically  significant. 

•  Test  Two,  conducted  in  Annapolis,  Maryland.  This  anecdotal  assessment  compared 
respondent  patterns  to  an  automated  telephone  information  system  versus  an  on-line 
bulletin  board  system  in  an  African  American  low-income  target  group. 

•  Test  Three,  conducted  in  Philadelphia,  Pennsylvania.  This  test  was  a  comparative 
assessment  of  caller  preferences  in  four  target  groups  for  an  automated  telephone 
information  system  versus  a  live  counselor  referral  service.  Test  Three  mirrored  the 
design  of  Test  One  in  a  different  geographic  area,  and  was  designed  to  validate  the 
statistically  significant  results  of  Test  One. 

•  Test  Four,  conducted  in  Baltimore,  Maryland.  This  anecdotal  test  compared  caller 
response  patterns  to  different  promotional  media  and  different  media  messages  in  an 
African  American  low-income  target  group. 


5 


V.  Body 


A  EXPERIMENTAL  METHODS  USED  AND  RESULTS  OBTAINED 

1.  Research  Method  and  Testing  Strategy 

As  stated  in  the  Introduction  section,  nine  major  tasks  were  identified  to  answer  the 
research  questions,  as  follows: 

Task  TDetermine  content  of  the  information  system 
Task  2:  Research  technical  system  design  issues 

Task  3:  Research  issues/topics  and  produce  messages  identified  for  the  information 
system’s  audiotex/faxiotex  library 
Task  4:  Acquire  and  set  up  the  demonstration  system  model 
Task  5:  Establish  system  and  data  retrieval  routines 

Task  6:  Develop  and  implement  the  outreach  and  promotional  campaign  plan 
Task  7:  Conduct  at  least  two  comparable  tests  of  the  system 
Task  8:  Research,  define,  and  identify  appropriate  future  modes  of  sustaining  the 
information  beyond  the  test  phase 
Task  9:  Document  and  report  results 

The  hypothesis  of  the  research  study  stated  that  “informationally  hard  to  reach” 
populations  will  prefer  an  automated  telephone  information  system  over  a  live  counselor 
systems  as  a  means  of  obtaining  breast  health  information.  The  major  hypothesis  and 
related  hypotheses  are  detailed  below. 

Major  Hypothesis:  Multimedia  automated  telephone  systems  enhance  live  referral  alone  as 
a  vehicle  to  reach  the  informationally  hard-to-reach. 

Related  Hypotheses: 

1 .  Callers  overall  will  show  a  greater  tendency  to  call  a  multimedia  automated  telephone 
system  combined  with  live  referral  than  to  call  the  live  referral  alone. 

2.  Within  racial  groups,  callers  at  different  socio-economic  status  (SES)  levels  will  show 
significant  differences  in  caller  preferences  for  use  of  a  multimedia  automated  telephone 
system  with  a  live  referral  option  compared  to  live  referral  alone  . 

3.  Callers  at  lower  SES  leyels  will  show  a  significant  preference  for  the  use  of  a 
multimedia  automated  telephone  system  with  a  live  referral  option  compared  to  live 
referral  alone. 
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4.  Callers  with  higher  SES  levels  will  show  significant  differences  by  racial  groups  in 
their  preference  for  the  use  of  a  multimedia  automated  telephone  system  with  a  live 
referral  option  compared  to  live  referral  alone. 

5.  Callers  with  lower  SES  across  racial  groups  will  show  a  preference  for  the  use  of  a 
multimedia  automated  telephone  system  with  a  live  referral  option  compared  to  live 
referral  alone. 

6.  A  culturally  sensitive  outreach  strategy  will  increase  caller  volume  compared  to  a  less- 
culturally-focused  outreach  strategy. 

7.  Callers  in  different  geographical  areas  will  show  a  significant  preference  for  a 
multimedia  automated  telephone  system  with  live  referral  compared  to  live  referral 
alone. 

Within  the  context  of  this  research  and  the  hypotheses,  the  technical  option  was  designed  to 
collect  data  related  to  six  issues: 

1 .  Number  of  callers  from  a  particular  area 

2.  Number  of  topics  accessed  by  a  caller,  general  preferences,  preferences  by  census  area, 
preferences  for  calls  taking  the  personal  assessment 

3 .  Number  of  callers  who  transferred  to  a  live  counselor  by  census  area 

4.  Analysis  of  caller  topic  preferences  by  call  census  area 

5.  Number  of  callers  requesting  fax  or  hard  copy  by  census  area 

6.  Number  of  callers  by  census  area  who  responded  to  follow-up  surveys 

To  test  the  hypothesis,  a  pre-test  and  four  tests  were  conducted  over  the  three-year 
research  period.  As  the  focus  of  the  study,  we  examined  user  preferences  for  an 
automated  telephone  information  system  versus  a  live  counselor  system  to  determine  if 
there  are  any  statistically  significant  differences  among  racial  and  income  groups. 

The  research  design  for  the  Breast  Health  Information  Project  included  two  anecdotal 
tests,  in  addition  to  a  comparative  testing  research  design.  The  comparative  test  design 
replicates  Test  One  and  Three  to  provide  large  sampling  sizes  in  different  geographical 
areas.  This  comparative  test  design  examined  the  validity  of  test  results.  However,  Test 
Two  and  Test  Four  were  established  to  provide  anecdotal  information  in  two  important 
areas  related  to  providing  health  information.  These  tests  examined  the  impact  of  a 
different  technology  and  different  outreach  strategies  in  reaching  the  informationally- 
hard-to-reach. 

TeleSonic's  three-year  study  was  built  upon  the  premise  that  an  innovative  automated 
telephone  information  system  could  increase  preventive  actions  against  breast  cancer 
among  minority  and  underserved  female  populations  by  fostering  them  to  take  action.  To 
measure  its  success  level  in  accomplishing  this  goal,  three  specific  "behavior  levels"  were 
defined.  Each  behavior  represented  a  different  level  of  preventive  or  proactive  action 
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taken  by  callers  to  the  automated  telephone  information  system.  TeleSonic  anticipated 
different  response  patterns  to  each  of  the  three  behavior  levels. 

Behavior  Level  One  measured  caller  response  rates  to  a  targeted,  controlled  direct  mail 
campaign.  For  this  measurement,  call  counts  into  the  telephone  system  were  tracked  and 
compared  to  the  total  number  of  pieces  mailed.  Because  this  was  the  most  basic  of  the 
three  behavior  levels,  the  greatest  response  level  was  expected  in  this  category. 

Behavior  Level  Two  measured  what  actions  callers  took  when  they  called  the  automated 
telephone  system.  Actions  in  this  behavior  category  indicated  how  actively  callers 
utilized  the  breast  cancer  prevention  information  presented.  For  this  behavior  level, 
TeleSonic  measured  caller  actions  such  as  time-on-line,  number  of  messages  listened  to, 
transfers  to  a  live  counselor,  requests  for  information  by  mail  or  fax,  completion  of  a 
"personal  profile"  assessment",  and  participation  in  an  optional  survey. 

Behavior  Level  Three,  the  most  difficult  category  to  measure,  examined  subsequent 
actions  of  callers  to  determine  if  they  took  any  new  preventive  and/or  proactive  breast 
cancer  prevention  actions  as  a  result  of  using  the  telephone  information  system.  Behavior 
Level  Three  actions  included  such  proactive  steps  as  practicing  breast  self-examination, 
scheduling  an  appointment  with  a  medical  practitioner,  or  scheduling  a  mammogram. 

The  first  two  behavior  levels  could  be  measured  directly,  based  on  tracking  and  analyzing 
caller  telephone  actions.  The  third  behavior  level  was  based  on  self-reports  from  callers 
to  the  system,  without  verification.  It  was  also  based  on  capturing  information  from  people 
volunteering  to  participate  in  follow  up  survey  efforts.  The  self  report  basis  for  Behavior 
Level  Three  reflects  one  of  the  limitations  of  this  research  effort. 

Data  capture  information  for  past  telecommunications  systems  have  basically  focused  on 
the  level  of  throughput  and  the  number  of  calls  received.  Therefore,  assessing  and 
comparing  response  patterns  for  Behavior  Levels  2  and  3  that  are  directly  tied  to  a 
telecommunications  application  became  both  a  challenge  and  a  limitation  for  this  research. 


Determination  of  Sample  Size 

In  developing  expected  caller  response  statistics,  information  on  Behavior  Level  1  response 
patterns  were  available  from  a  range  of  application  telephone  information  systems. 
Response  patterns  were  included  from  job  lines,  real  estate  areas,  travel  information,  dating 
services,  health  services,  and  retirement  services. 

Based  on  response  patterns  for  other  telecommunications  applications,  in  general,  the 
estimated  calls  to  the  live  counselor  system  were  originally  projected  at  5%  for  the  first 
month,  and  up  to  7.5%  for  a  three-month  period.  A  higher  response  pattern  was  projected 
for  the  automated  telephone  system,  at  7%  for  the  first  month,  and  up  to  10%  for  a  three- 
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month  period.  These  response  rates  are  achievable  when  appropriate  advertisements  and 
promotions  are  used  to  market  the  information  systems. 

Because  these  expectation  levels  were  met  and  exceeded  during  the  pretest,  the  adequacy  of 
sampling  size  was  calculated  for  each  treatment  group  to  be  tested.  Actual  response  rates  in 
testing  were  much  lower,  however.  As  a  consequence,  the  adequacy  of  the  required  sample 
sizes  for  each  treatment  group  was  recalculated  to  determine  any  need  for  expanding  the  test 
population.  The  recalculation  showed  that  the  size  of  the  response  level  was  appropriate  for 
statistical  analyses  for  Behavior  Level  1 .  Response  sizes  were  inappropriate  for  statistical 
analyses  for  Behavior  Levels  2  and  3.  However,  qualitative  measurements  were  possible 
for  these  two  behavior  levels.  In  addition  to  using  Chi-square  tests  for  quantitative 
statistical  analyses,  qualitative  analyses  were  provided  through  summary  notes  of  data 
obtained  from  the  research  and  reactions  from  focus  group  assessments.  The  focus  group 
reactions  to  the  results  produced  a  representative  assessment  of  the  validity  of  the  data 
response  for  the  appropriate  targeted  population. 


COMPARATIVE  TESTS:  TESTS  ONE  AND  THREE 
TEST  ONE 

Both  Tests  One  and  Three  were  designed  to  assess  caller  preferences  in  four  target  groups 
for  an  automated  telephone  information  system  versus  a  live  counselor  referral  service. 
For  each  test,  a  geographic  test  area  was  selected  based  on  "qualifying"  census  data. 
Because  the  testing  strategy  involved  mailing  information  to  four  distinct  target  groups 
and  then  measuring  the  response  rates,  the  test  area  needed  to  have  "concentration  areas" 
of  each  target  group.  To  ensure  the  statistical  validity  of  results,  a  minimum 
concentration  level  of  the  target  population  was  set  for  qualifying  census  areas.  For 
example,  a  census  area  defined  as  "African  American  Lower  Moderate"  needed  to  have  at 
least  a  55%  minimum  percentage  of  African  American  Lower  Moderate  persons  living  in 
that  census  area.  The  minimum  percentage  was  identified  to  ensure  the  overall 
concentration  of  our  distinct  target  groups  to  support  call  level  expectations. 

Once  the  test  area  was  qualified  and  selected,  TeleSonic  created  "maps"  for  each  of  the 
four  target  groups  using  Geographic  Information  System  (GIS)  mapping  software.  These 
maps  identified  the  qualifying  Census  Block  Groups  (CBGs)  within  the  test  area  that  met 
the  minimum  concentration  level  or  higher  for  the  particular  target  group. 

After  identifying  the  "qualifying"  CBGs  for  each  target  group,  TeleSonic  then  utilized  a 
mailing  house  to  randomly  select  5,000  addresses  for  each  target  group.  Each  of  the  four 
target  groups  (WLM,  MWU,  AALM,  and  AAMU)  was  divided  in  half,  for  a  total  of  eight 
mailing  groups.  Each  mailing  group  consisted  of  2,500  households.  For  tracking 
purposes,  eight  promotional  pieces  or  "postcards"  were  created  to  correspond  to  the  eight 
mailing  groups,  each  displaying  a  different  800-telephone  number.  Four  of  the  eight 
postcards  promoted  the  automated  telephone  information  system,  and  four  promoted  a 
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live  counselor  referral  service.  Under  controlled  conditions,  2,500  randomly  selected 
households  in  each  of  the  eight  mailing  groups  were  then  mailed  a  postcard,  and  caller 
response  rates  were  tracked. 


Table  1:  Direct  Mail  Design  Strategy  —  Test  One  (Baltimore,  MD)  and  Test  Three 
(Philadelphia,  PA) _ 


TARGET  GROUP 

NUMBER  OF 

AUTOMATED 

TELEPHONE 

INFORMATION 

SYSTEM 

POSTCARDS 

MAILED 

NUMBER  OF 

LIVE 

COUNSELOR 

SYSTEM 

POSTCARDS 

MAILED 

TOTAL  NUMBER 
OF  POSTCARDS 
MAILED 

WLM 

2,500 

2,500 

5,000 

WMU 

2,500 

2,500 

5,000 

AALM 

2,500  • 

2,500 

5,000 

AAMU 

2,500 

2,500 

5,000 

TOTALS 

10,000 

10,000 

20,000 

Test  One  was  conducted  in  the  Baltimore  metropolitan  area  in  the  Fall  of  1995.  Based 
on  call  volumes  into  the  live  and  automated  systems,  as  well  as  caller  activity  within 
those  systems,  we  were  able  to  determine  some  caller  preferences  by  time  of  day,  race, 
and  income  level.  Test  Three  was  conducted  in  the  Fall  of  1996  in  the  Philadelphia 
metropolitan  area.  Test  Three  was  designed  to  validate  the  Test  One  results  in  a  different 
geographic  area,  using  higher  racial  and  income  density  criteria  levels. 

Test  One  was  implemented  during  the  three-month  period  of  September  14  through 
December  17,  1995.  As  previously  noted,  the  basic  design  included  the  mailing  of 
postcards  to  20,000  households  across  the  metropolitan  Baltimore,  Maryland  area.  The 
mailings  were  randomly  assigned  across  a  total  of  634,750  households,  affecting 
Baltimore  City  and  the  five  central  Maryland  counties  of  Anne  Arundel,  Baltimore, 
Carroll,  Harford,  and  Howard. 

Within  the  affected  households,  the  populations  were  segmented  into  four  census  area 
groups,  each  with  two  sub-groups.  The  four  census  areas  included  low  to  moderate 
African  Americans  and  whites  and  middle  to  upper  income  census  areas  for  the  same 
racial  groups.  Each  of  the  four  groups  were  then  further  segmented  by  random 
assignment  into  Treatment  A  (households  to  receive  the  postcard  about  the  live  counselor 
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system)  and  Treatment  B  (households  to  receive  the  postcard  about  the  automated 
information  system). 

To  control  and  monitor  data  results,  postcards  were  printed  with  a  separate  telephone 
number  for  each  of  the  eight  treatment  groups.  This  process  allowed  research  staff  to 
capture  and  segment  caller  activity  by  census  area  and  treatment  group.  As  a 
consequence,  eight  separate  and  unique  telephone  numbers  were  dedicated  to  this  test, 
each  for  a  specific  treatment  group.  Caller  activity  could  then  be  analyzed  by  income  and 
race. 

Identification  of  Households  -The  random  assignment  of  the  four  groups  into  Treatment 
A  and  Treatment  B  was  made  after  receipt  of  address  labels  to  support  the  mailings  to  the 
specific  addresses.  This  was  accomplished  by  identifying  a  mail  house  that  could  provide 
address  labels  with  the  highest  match  to  the  number  of  households  identified  within  the 
metropolitan  Maryland  census  analyses.  The  two  mail  houses  with  the  closest  match  in 
household  counts  were  then  assessed  to  determine  their  ability  to  provide  geo-coded 
address  labels  to  identify  households  within  specific  census  blocks.  Specific  directions 
were  given  to  the  mail  house  regarding  the  selection  process  for  the  address  labels.  The 
key  directions  included  five  major  steps: 

1.  Using  census  block  group  files  to  create  the  four  defined  census  areas,  which 
represent  the  qualifying  addresses. 

2.  Assigning  a  random  number  to  the  addresses,  using  a  random  program 
generator. 

3.  Drawing  20,000  random  addressees,  5,000  from  each  census  area  1  to  4,  in 
randomly  selected  groups  A  and  B. 

4.  Assembling  the  eight  sets  of  random  census  areas  in  zip  code  order. 

5.  Applying  Cheshire  labels  and  clearly  marking  the  census  sets  for  each  of  the 
four  census  areas  as  census  lists  1 A  through  4B. 

Once  the  address  labels  were  received,  a  blind  assessment  was  completed  to  determine 
the  accuracy  of  the  address  information.  The  blind  test  included  assessment  of  two  to 
three  11  x  15  sheets  of  addresses  from  each  of  the  four  targeted  groups.  The  address 
labels  were  received  with  each  of  the  four  targeted  groups  labeled  as  a  separate  package 
of  labels.  Each  of  the  four  packages  was  broken  into  two  sets  of  addresses:  sets  “A”  and 
“B,”  corresponding  with  Treatment  A  and  Treatment  B. 

To  assess  the  accuracy  of  the  address  labels,  a  page  of  address  labels  was  taken  from  near 
the  beginning,  the  middle  or  the  end  of  each  group.  These  were  copied  and  all  sheets  were 
then  mixed  together  without  labels.  Appropriate  research  staff  were  then  asked  to 
complete  a  demographic  comparison  by  comparing  the  addresses  from  the  sheet  with  the 
census  block  information  to  determine  which  of  the  four  census  groups  that  the  sheets 
represented.  This  blind  test  provided  verification  that  the  addresses  were  appropriately 
labeled  as  low  to  moderate  income  African  American  or  White,  and  as  middle  to  upper 
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income  African  American  or  White.  Once  the  address  labels  were  verified,  each  set  of 
four  labels  was  affixed  to  the  postcards  for  mailing. 


Mailing  Process  -The  Phase  One  Test  was  designed  to  obtain  responses  over  a  three- 
month  period,  based  on  the  expected  response  rates  for  telecommunication  systems.  The 
mailing  to  the  20,000  households  was  spread  in  equal  increments  over  an  extended  period 
of  three-week  intervals,  to  ensure  that  no  system  overload  occurred  for  either  CIS  or 
ACS.  Each  treatment  group  within  the  four  target  areas  received  625  mailings  per 
interval,  for  a  total  mailing  of  5,000  per  target  area,  based  on  the  following  schedule: 


1st  mailing: 
2nd  Mailing: 
3rd  Mailing: 
4th  Mailing: 


September  13, 1995 
October  6, 1995 
October  26, 1995 
November  17,  1995 


625  mailings  each  treatment  Total  mailings  5,000 
625  mailings  each  treatment  Total  mailings  5,000 
625  mailing  each  treatment  Total  mailing  5,000 
625  mailings  each  treatment  Total  mailing  5,000 


The  mail  was  processed  through  a  two-phase  effort.  First,  research  staff  prepared  eight 
different  sets  for  mailing.  These  sets  included  the  postcard  emd  the  appropriate  address 
labels  for  each  treatment  group.  A  second  staff  member  was  assigned  to  verify  the 
accuracy  of  the  first  staff  person’s  efforts.  The  sets  were  placed  into  eight  different 
mailbags  and  taken  to  a  local  mail  house  for  mail  preparation  and  final  mailing. 

The  local  mail  house  was  responsible  for  affixing  address  labels  to  the  postcard,  sorting 
postcards  by  zip  code,  affixing  stamps,  then  delivering  the  eight  mailbags  to  the  central 
post  office  for  mailing.  Since  a  mail  house  was  responsible  for  a  portion  of  the  mailing 
effort,  research  staff  monitored  the  mail  process  to  determine  actual  dates  the  postcards 
were  delivered  to  the  US  Postal  Service.  The  mailing  dates  noted  earlier  represent  the 
actual  dates  the  mailbags  were  submitted  to  the  US  Postal  Service. 


All  postcards  were  mailed  bulk  rate,  address  correction  requested.  The  delivery  date  for 
bulk  mail  in  the  target  area  ranged  from  one  to  ten  days,  as  reported  by  the  postal  service. 
The  “address  correction  requested”  stamp  was  used  to  increase  the  level  of  assurance  that 
undelivered  postcards  were  returned,  and  therefore  could  be  analyzed  to  determine 
delivery  rates  and  any  major  variance  by  treatment  groups. 


The  expected  response  rates  were  based  on  the  assumption  of  responses  over  a  three- 
month  period  of  exposure.  Under  this  assumption,  the  close  out  period  for  data  collection 
should  have  gone  until  three  months  after  the  final  mailing,  to  February  17th.  The  data 
collection,  however,  was  contained  within  a  data  collection  period  of  three  months  from 
the  first  mailing,  September  13  through  December  17th.  After  data  analyses,  the  data 
collection  period  was  then  augmented  by  focus  group  reviews. 
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Postcards  were  mailed  to  households  designated  as  either  Treatment  A  or  Treatment  B. 
Treatment  A  households  received  a  postcard  that  promoted  the  opportunity  to  speak  to  a 
live  counselor  and  get  answers  to  individual  questions.  (A  sample  postcard  for  Treatment 
A,  Live  counselor  information  system,  is  included  in  Appendix  B).  The  Treatment  B 
households  received  a  postcard  that  promoted  the  convenience  of  calling  an  information 
system,  seven  days  a  week,  24  hours  a  day.  The  postcard  for  Treatment  B  provided 
information  about  a  “library”  of  prerecorded  message  topics  that  covered  issues  about 
breast  health.  (A  sample  postcard  for  Treatment  B,  automated  information  system  is 
included  in  Appendix  C). 

Data  Collection  -  An  additional  area  of  activity  during  the  test  period  related  to  data 
collection.  Caller  activity  was  analyzed  by  examining  caller  "logs".  These  caller  logs 
provided  identification  of  the  telephone  number  being  called  (corresponding  to  census 
area  and  treatment  group),  time  and  date  of  call,  length  of  call,  and  the  different  features 
used  by  callers.  This  information  was  captured  for  both  the  automated  and  the  live 
counselor  information  systems.  As  a  note,  however,  once  the  caller  to  the  live  counselor 
transferred  from  the  research  system  to  CIS  or  ACS,  the  system  could  no  longer  monitor 
caller  activity.  Whether  or  not  the  caller  was  successful  in  accomplishing  the  call  or 
experienced  a  busy  signal  could  not  be  assessed.  As  a  consequence,  the  prerecorded 
system  messages  gave  each  caller  choosing  to  call  CIS  or  ACS  the  direct  telephone 
numbers  in  the  event  that  a  busy  signal  was  encountered. 

Data  on  the  call  behavior  by  treatment  group  was  collected  automatically  by  a  call  log.  It 
should  be  noted  that  the  originating  phone  number  of  the  call  was  not  tracked,  to  protect 
the  identity  of  callers.  The  only  identifying  information  came  from  those  callers  who 
voluntarily  offered  to  provide  follow-on  feedback.  Computer  program  routines  were 
written  to  purge  data  and  generate  reports  on  variables  needed  for  research  analysis.  Data 
were  purged  to  exclude  staff  calls  for  daily  line  qualification,  review  and  system 
evaluation.  These  calls  were  coded  for  easy  extraction.  In  addition,  computerized  calls 
into  the  system  were  purged  based  on  computer  programming  patterns  that  were 
discerned.  These  included  set  patterns  of  seven  calls  to  the  same  telephone  number  at  six 
to  seven  second  intervals.  Call  reports  were  also  monitored  to  ensure  that  no  calls  were 
received  from  areas  blocked  from  the  call  region. 

Finally,  calls  were  excluded  for  the  four-day  period  of  November  28th  through  December 
1st  for  all  treatment  groups.  These  calls  were  excluded  because  of  a  highly  atypical  call 
pattern  that  was  observed  within  one  treatment  group.  The  calls  were  outside  of  any 
normal  range  of  calls  for  the  . treatment  group.  During  this  period,  the  daily  average  call 
was  nearly  three  to  four  times  the  daily  average  during  any  other  period.  This  occurred 
during  a  period  when  no  mailings  were  recently  delivered  by  the  postal  service,  when 
there  were  no  national  newsworthy  announcement  affecting  breast  health  or  breast  cancer. 
Some  of  the  calls  could  be  tracked  and  purged  as  a  result  of  staff  testing  but  others  were 
still  questionable.  As  a  consequence,  to  ensure  consistency  of  effort,  calls  were 
eliminated  from  all  treatment  groups  for  the  period. 
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To  ensure  efficient  functioning  of  the  system,  during  the  three-month  period,  daily  line 
qualification  tests  were  conducted.  These  line  qualification  tests  required  that  staff  check 
the  system  to  ensure  that  all  lines  were  functioning,  to  check  and  identify  any  disruptive 
sounds  or  line  interference,  and  to  check  the  volume  levels.  Only  minor  interferences 
were  identified,  and  corrected.  No  problems  encountered  during  this  period  were 
significant  enough  to  skew  results  from  any  treatment  groups. 

As  a  final  note,  the  system  operated  on  an  “automatic  pilot”  during  the  three-month 
implementation  period.  Once  the  system  was  developed  and  activated,  all  activities  were 
directed  toward  monitoring  the  system  for  operational  functionality,  periodic  mailing,  and 
data  collection. 

Data  Analysis  -  Quantitative  and  qualitative  assessments  were  used  to  analyze  the  data 
received  from  these  two  tests.  The  Chi-square  test,  which  is  both  a  test  of  association  and 
distribution,  was  used  to  provide  the  quantitative  assessments.  The  test  compares  the 
frequency  of  observed  and  expected  occurrences  for  the  data  samples.  This  statistical  test 
measures  whether  the  differences  observed  are  sufficiently  large  enough  to  conclude  that 
they  did  not  occur  by  chance,  but  instead  reflect  real  differences.  The  Chi-square  test  shows 
the  level  of  association  between  two  categorical  variables.  In  addition,  it  can  be  used  to 
show  the  strength  of  that  association. 

A  probability  level  of  p  <  .05  was  used  to  investigate  the  strength  of  the  association  between 
variables.  The  research  results  were  analyzed  under  the  statistical  leadership  of  a  statistical 
consultant  from  the  University  of  Maryland  Hospital  in  Baltimore,  Maryland. 


Focus  Group  Review  -Focus  group  discussions  were  held  during  the  development  and 
pretest  of  the  BHIP  research  effort.  These  discussions  proved  invaluable  in  developing 
the  mailing  process  and  contents  and  in  modifying  system  functions  and  content.  As  a 
consequence,  when  the  review  of  the  analyzed  data  showed  overwhelming  caller 
preferences  for  the  automated  system,  it  was  concluded  that  additional  focus  groups 
might  be  helpful  in  gaining  insight  into  the  call  behavior. 

During  the  months  of  February,  March,  and  April  1996,  five  focus  group  sessions  were 
identified  and  held.  These  groups  were  identified  by  working  with  central  Maryland 
organizations  whose  memberships  are  representative  of  the  four  census  areas  in  our  target 
groups.  Originally,  four  focus  group  sessions  were  planned.  TeleSonic  requested  each 
group  participant  to  voluntarily  complete  a  pre-questionnaire  before  discussions.  This 
pre-questionnaire  was  designed  to  determine  any  changes  in  opinions  and  attitude,  as  a 
result  of  the  discussions  and  to  ensure  appropriate  income  representation.  The  pre¬ 
questionnaire  showed  that  low-income  representation  was  not  being  provided  from  the 
expected  groups,  as  a  consequence,  an  additional  focus  group,  for  a  total  of  five,  was 
identified  and  conducted. 
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Each  group  meeting  took  place  during  a  lunch  period,  or  just  before  or  after  lunch  and 
included  an  opportunity  for  general  discussions  over  refreshments,  before  the  discussion 
commenced.  Each  group  followed  the  same  process,  which  included: 

1.  Introductions 

2.  Purpose 

3 .  Summary  of  Phase  One  Test  results 

4.  Completion  of  pre-questionnaire 

5.  An  opportunity  to  listen  to  the  automated  messages  and  review  the  postcards, 
and 

6.  Discussion  of  six  questions. 

The  focus  group  sessions  lasted  from  one  and  one-half  to  two  hours,  on  average. 
Members  were  asked  if  they  would  agree  to  have  sessions  recorded  to  ensure  accuracy  in 
reporting;  all  participants  agreed. 

Test  One  Findings 

Overall,  the  findings  fi'om  Test  One  research  efforts  showed  a  significant  difference  in  the 
response  pattern  of  callers,  as  follows: 

•  Caller  results  showed  a  preference  for  the  automated  information  system  compared  to 
the  live  counselor  information  system. 

•  Callers  across  all  race  and  income  lines  preferred  using  the  automated  system  in  the 
evenings,  from  6PM  to  12AM. 

•  There  were  no  significant  differences  in  response  by  race. 

•  Significant  differences  were  observed  between  income  groups. 

By  race,  African  Americans  and  Whites  responded  very  similarly  to  the  mailing.  Similar 
numbers  of  African  Americans  and  Whites  called  the  automated  and  live  counselor 
systems.  The  only  discernible  racial  difference  was  that  African  Americans  spent  more 
time  using  the  automated  system  than  Whites. 

By  income  level,  a  greater  number  of  low  to  moderate  income  level  persons  responded  to 
the  mailing,  called  the  automated  system,  and  spent  more  time  using  the  automated 
system  than  middle  to  upper  income  level  persons. 

The  number  of  telephone  calls  from  the  African  American  and  White  census  areas  were 
about  the  same,  a  little  better  than  300  each.  Overall,  650  calls  (a  response  level  of  3.3%) 
were  received.  Although  the  response  level  is  less  than  originally  anticipated,  it  was  twice 
as  high  as  the  response  pattern  of  approximately  2%  that  is  considered  good  from  direct 
mailing,  and  is  large  enough  to  assess  areas  of  significance  between  treatment  groups. 

During  the  original  design  and  planning  stage,  it  was  acknowledged  that  a  mail  outreach 
strategy  without  any  related  outreach  support  would  not  be  as  effective  as  other  more 
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aggressive  or  direct  outreach  efforts.  However,  it  was  concluded  that  a  mail  outreach 
strategy  would  provide  the  cleanest  design  for  clearly  assessing  any  difference  in  responses 
for  the  targeted  populations.  As  a  consequence,  to  control  for  seepage,  the  general  use  of 
advertising  and  promotion  was  eliminated. 

The  number  of  calls  from  lower  income  census  blocks  were  significantly  higher  than  those 
calls  coming  from  middle  to  upper  income  census  blocks  (376  compared  to  274).  This 
pattern  was  observed  within  the  two  racial  groups.  One  hundred  ninety-five  low  to 
moderate  African  Americans  responded  to  the  mailing,  as  opposed  to  133  middle  to  upper 
income  African  Americans.  One  hundred  eighty-one  low  to  moderate  income  Whites 
responded  to  the  mailing,  compared  with  141  middle  to  upper  income  Whites. 

Tables  2, 3, 4, 5,  and  6  on  the  following  pages  summarize  these  results.  These  tables  show 
that  significant  results  exist  in  two  areas:  (1)  preference  for  the  automated  telephone  system, 
and  (2)  response  pattern  from  low  to  moderate  income  groups.  More  detailed  data  reports 
are  presented  in  Appendix  D,  Test  One  Data  Reports. 


Table  2:  Test  1  Comparison  of  Overall  Call  Responses 


Exposure 

Mailings 

Calls 

Response 

X  ^(p-value) 

Automated 

10,000 

406 

4.1% 

4i;7(p<o.odi) 

Live 

i  0,000 

244 

2.4% 

Total 

20,000 

650 

3.3% 

Note:  Shaded  area  reflects  statistically  significant  differences. 
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Table  3:  Test  1  Comparison  of  Call  Responses  by  Income  and  Racial  Groups 


Exposure 

Mailings 

Calls 

Response 

X  ^(p-value) 

African 

10,000 

328 

3.3% 

0.057(p=0.811) 

American 

White 

10,000 

322 

3.2% 

Low-Moderate 

10,000 

376 

3.8% 

1^:5(p#T001) 

Middle  -Upper 

10,000 

274 

2.7% 

African 

American 

Low-Moderate 

2,500 

195 

3.9% 

i2;i^<d.Mi|., 

Middle-Upper 

2,500 

133 

2.7% 

White 

Low-Moderate 

2,500 

181 

3.6% 

^5.i(^d.'02) ; ,  ’ 

Middle-Upper 

2,500 

141 

2.8% 

Low  -Moderate 

African 

2,500 

195 

3.9% 

;0.542(p=d.462) 

American 

White 

2,500 

181 

3.6% 

Middle-Upper 

African 

2,500 

133 

2.7% 

0.24(p=0.624) 

American 

White 

2,500 

141 

2.8% 

Note:  Shaded  areas  reflect  statistically  significant  differences. 


Table  4:  Test  1  Comparison  of  Call  Responses  by  Treatment  within  Racial  and 
Income  Groups 


Exposure 

Mailings 

Calls 

Response 

X  ^(p-value) 

African  American 

Automated 

5000 

200 

4.0% 

16.3(p<0.W)l), 

Live 

5000 

128 

2.6% 

White 

Automated 

5000 

206 

4.1% 

:26;0(p<0.001). 

Live 

5000 

116 

2.3% 

Low-Moderate 

Automated 

5000 

231 

4.6% 

20.4jj)<0.001) 

Live 

5000 

145 

2.9% 

Middle-Upper 

Automated 

5000 

175 

3.5% 

?g;:7(p<p.ooi)'~ 

Live 

5000 

99 

2.0% 

Low-Moderate 

African  American 
Automated 

2500 

114 

4.6% 

5:81  (p^).016) 

Live 

2500 

81 

3.2% 

White 

Automated 

2500 

117 

4.7% 

Live 

2500  ■ 

64 

2.6% 

Middle-Upper 

African  American 
Automated 

2500 

86 

3.4% 

Jll;7(H-60lC  ' 

Live 

2500 

47 

1.9% 

White 

Automated 

2500 

89 

3.6% 

Live 

2500 

52 

2.1% 

Note:  Shaded  areas  reflect  statistically  significant  differences. 


Table  5:  Test  1  Comparison  of  Call  Responses  to  the  Automated  Information  System 


Exposure 

Mailings 

Calls 

Response 

x^  (p-value) 

African-American 

5000 

200 

4.0% 

0.09  (p=0.761) 

White 

5000 

206 

4.1% 

V  . . . 

Low-Moderate 

5000 

231 

4.6% 

g.osM-Q'&lv 

Middle-Upper 

5000 

175 

3.5% 

Low-Moderate 

African- 

2500 

114 

4.6% 

0.041  (p=0.84) 

American 

White 

2500 

117 

4.7% 

Middle-Upper 

African- 

2500 

86 

3.4% 

0.053  (p=0.817) 

American 

White 

2500 

89 

3.6% 

African-American 

Low-Moderate 

2500 

114 

4.6% 

:ib8ii?=o;643):.~::' 

Middle-Upper 

2500 

86 

3.4% 

:■?' 

White 

Low-Moderate 

2500 

117 

4.7% 

Middle-Upper 

2500 

89 

3.6% 

Note:  shaded  areas  reflect  statistically  significant  differences. 


18 


Table  6:  Test  1  Comparison  of  Call  Responses  to  the  Live  Counselor  Information 
System 


Exposure 

Mailings 

Calls 

Response 

x2  (p-value) 

African-American 

5000 

128 

2.6% 

0.605  (p=0.437) 

White 

5000 

116 

2.3% 

. . 

Low-Moderate 

5000 

145 

2.9% 

8;89  (1^.003) 

Middle-Upper 

5000 

99 

2.0% 

"  '.""'I'-' 

Low-Moderate 

African- A  merican 

2500 

81 

3.2% 

2.05  (p=0.152) 

White 

2500 

64 

2.6% 

Middle-Upper.- 

African- A  merican 

2500 

47 

1.9% 

0.258  (p=  0.612) 

White 

2500 

52 

2.1% 

African-American 

Low-Moderate 

2500 

81 

3.2% 

:9.27(p-0.002r“^ 

Middle-Upper 

2500 

47 

1.9% 

White 

Low-Moderate 

2500 

64 

2.6% 

1.27  (p=0.26) 

Middle-Upper 

2500 

52 

2.1% 

Note:  shaded  areas  reflect  statistically  significant  differences. 

In  addition  to  the  information  on  call  volume  patterns,  some  additional  results  were 
observed  from  the  Test  One  research  effort.  These  include  caller  preference  related  to  time 
of  day,  specific  messages  accessed,  and  postcard  return  patterns. 

Timing  of  Calls:  Evenings  and  Sundays  were  the  peak  calling  times  for  the  automated 
system,  whereas  Wednesday  evenings  was  the  peak  calling  period  to  the  live  counselors. 
Callers  from  the  African  American  census  groups  used  Monday  and  Wednesday  evenings 
as  their  peak  call  times  for  the  automated  system,  and  Thursday  afternoons  and  Wednesday 
evenings  as  their  peak  call  times  to  the  live  counselor.  Callers  from  the  white  census  areas 
preferred  weekend  evenings  for  calls  to  the  automated  information  system  and  Wednesday 
evening  and  Friday  afternoons  as  their  peak  call  periods  to  the  live  counselor. 

Except  for  callers  from  the  low  to  moderate  African  American  income  area,  caller 
preferences  were  for  evening  calls.  Low  to  moderate  income  African  American  census 
areas  showed  call  time  preferences  of  Thursday  afternoon  for  the  live  counselor  system. 
There  was  much  variation  in  the  preferred  call  days  with  no  observable  consistent  pattern 
beyond  the  evening  preferehces,  except  as  previously  noted.  Clearly,  the  call-time 
preferences  indicate  obvious  considerations  for  when  users  are  most  likely  to  access 
telecommunications  information  systems. 
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Table  7  summarizes  the  high/low  call  periods  for  each  treatment  group. 
Table  7:  Test  1  High  and  Low  Calling  Periods 


Stratesv 

Income 

Level 

Call  Periods 

AALM 

AAM 

u 

WLM 

WMU 

M 

w 

LM 

MU 

Overall 

Call  Volume  Peak  - 

Mon. 

Wed. 

Sun. 

Sun. 

M 

Sun. 

Sun. 

Wed. 

Evenings 

Overall 

Eve. 

Eve. 

Eve 

Eve. 

&W 

Eve. 

Eve. 

Eve. 

Eve. 

Lowest  Call  Volume- 

Night 

Night 

Night 

Night 

Tue,F 

Night 

Tue. 

Night 

Tue.  & 

Overall 

&Sat 

& 

& 

Sat. 

night 

Sun. 

Sat. 

nights 

mom. 

night 

Peak-  Auto 

Mon. 

Wed. 

Sun.,  F 

Sun 

Wed. 

Sun 

Sun. 

Sun 

Evenings 

Eve. 

Eve. 

&Sat 

Eve  & 

Eve. 

Eve 

Eve 

& 

Eve. 

MAft. 

& 

Fri. 

M 

Eve. 

Aft. 

Call  Volume  Peak-  Live 

Thurs. 

Wed. 

WEve 

Wed. 

Fri. 

Wed. 

Mon. 

Fri. 

Aft.  • 

Eve. 

&  Fri 

Eve. 

Mom 

Eve 

Eve. 

Mom 

mnn 

Aft 

HU 

Lowest  C  Volume- Auto 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Lowest  C  Volume-Peak 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Definition  of  time  periods:  Morning  =  6am  -  12pm;  Afternoon  =12pm-6pm;  Evenings=6pm-12am; 
Night=12am-6am 

Strategy  Abbreviations:  AA=  African  American  Census  Area;  W=White  Census  Area;  LM=low  to  moderate 
income  group;  MU=Middle  to  upper  income  group 


TEST  THREE 

Test  Three  was  designed  to  validate  the  Test  One  results  in  a  different  geographic  area, 
using  higher  racial  and  income  density  criteria  levels,  and  the  same  test  design.  In 
between  these  two  tests,  a  smaller  comparative  test  was  conducted  assessing  user 
preferences  for  an  automated  telephone  service  versus  an  on-line  computer  service. 
However,  because  Tests  One  and  Three  were  similar  in  design.  Test  Three  will  be 
discussed  in  this  section  prior  to  Test  Two. 

Test  Three  was  implemented  during  the  three-month  period  of  September  3  through 
December  8,  1996,  in  the  Philadelphia  metropolitan  area.  Because  the  test  was  designed 
to  validate  the  Test  One  results  in  a  different  geographic  area,  the  basic  design  mirrored 
Test  One.  Postcards  were  mailed  to  20,000  households  across  the  metropolitan 
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Philadelphia,  Pennsylvania  area.  The  mailings  were  randomly  assigned  across  households 
in  the  five  counties  of  Bucks,  Chester,  Delaware,  Montgomery,  and  Philadelphia. 

As  in  Test  One,  within  the' affected  households,  the  populations  were  segmented  into  four 
census  area  groups,  each  with  two  sub-groups.  The  four  census  areas  included  low  to 
moderate  African  Americans  and  whites  and  middle  to  upper  income  census  areas  for  the 
same  racial  groups.  Each  of  the  four  groups  were  then  further  segmented  by  random 
assignment  into  Treatment  A  (households  to  receive  the  postcard  about  the  live  counselor 
system)  and  Treatment  B  (households  to  receive  the  postcard  about  the  automated 
information  system). 

Identification  of  Households  -After  receipt  of  the  address  labels  from  the  mailing  house, 
the  four  groups  were  randomly  assigned  into  Treatment  A  and  Treatment.  The  mail  house 
utilized  for  Test  One  was  selected  again,  based  on  their  ability  to  provide  address  labels 
with  the  highest  match  to  the  number  of  households  identified  within  the  Pennsylvania 
census  analyses  and  their  ability  to  provide  geo-coded  address  labels.  The  mail  house 
utilized  the  same  procedure  as  in  Test  One  for  selecting  the  address  labels.  As  in  Test 
One,  once  the  address  labels  were  received,  a  blind  assessment  was  completed  to 
determine  the  accuracy  of  the  address  information. 

Mailine  Process  -  We  determined  from  caller  traffic  patterns  in  Test  One  that  no  system 
overloads  had  occurred  for  either  the  Cancer  Information  Service  (CIS)  or  the  American 
Cancer  Society  (ACS).  As  a  result,  in  Test  Three  all  20,000  pieces  were  mailed  at  once, 
rather  than  in  staggered  increments.  The  mailing  to  the  20,000  households  occurred  on 
September  3, 1996,  and  results  were  measured  through  December  8,  1996.  The  mail  was 
processed  through  the  same  two-phase  effort  utilized  in  Test  One. 

Data  Collection  -  As  in  Test  One,  caller  activity  was  analyzed  by  examining  caller 
"logs".  These  caller  logs  provided  identification  of  the  telephone  number  being  called 
(corresponding  to  census  area  and  treatment  group),  time  and  date  of  call,  length  of  call, 
and  the  different  features  used  by  callers.  To  preserve  caller  anonymity,  caller  logs  did 
not  provide  any  identification  of  the  callers  or  the  telephone  number  from  which  the  call 
was  placed.  To  ensure  efficient  functioning  of  the  system  during  the  three-month  period, 
daily  line  qualification  tests  were  conducted. 

Data  Analysis  -  Quantitative  and  qualitative  assessments  were  used  to  analyze  the  data 
received  from  these  two  tests.  The  Chi-square  test  was  used  to  provide  the  quantitative 
assessments.  The  test  compares  the  frequency  of  observed  and  expected  occurrences  for  the 
data  samples.  A  probability  level  of  p  <  .05  was  used  to  investigate  the  strength  of  the 
association  between  variables.  The  research  results  were  analyzed  under  the  statistical 
leadership  of  a  statistical  consultant  from  the  University  of  Maryland  Hospital  in  Baltimore, 
Maryland. 
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Test  Three  Findings 


The  findings  from  Test  Three  research  efforts  indicated  an  overall  preference  for  the 
automated  system.  In  addition,  Test  Three  results  provided  many  statistically  significant 
preferences,  as  follows: 

•  By  race,  African  Americans  showed  a  statistically  significant  preference  for  the 
automated  system. 

•  By  income  level,  lower  moderates  showed  a  statistical  preference  for  the  automated 
system.  This  statistically  result  occurred  in  both  Tests  One  and  Two. 

•  By  race  and  income.  White  lower  moderates  showed  a  statistically  significant 
preference  for  the  automated  system.  In  addition.  White  middle  upper  income  persons 
showed  a  statistically  significant  preference  for  the  live  counselor  system. 

In  addition  to  these  statistically  significant  findings,  callers  across  all  race  and  income 
lines  preferred  using  the  automated  system  in  the  evenings,  from  6PM  to  12AM.  Test 
Three  showed  no  statistically  significant  differences  in  the  response  rates  of  callers  to  the 
automated  and  live  counselor  systems.  By  race,  more  Whites  responded  to  the  mailing. 

By  income  level,  a  greater  number  of  middle  to  upper  income  level  persons  responded  to 
the  mailing,  in  particular  to  the  live  system. 

Tables  8, 9, 10, 11,  and  12  on  the  following  pages  summarize  the  Test  Three  results.  These 
tables  show  that  significant  results  exist  in  three  areas:  (1)  preference  of  African  Americans 
for  the  automated  telephone  system;  (2)  preference  of  low  to  moderate  income  groups  for 
the  automated  system;  and  (3)  Preference  by  White  lower  moderate  person  for  the 
automated  system.  More  detailed  data  reports  are  presented  in  Appendix  E,  Test  Three 
Data  Reports. 


Table  8:  Test  3  Comparison  of  Overall  Call  Responses 


Exposure 

Mailings 

Calls 

Response 

X  ^(p-value) 

Automated 

10,000 

396 

3.96% 

1.31  (p=0.25) 

Live 

10,000 

365 

3.65% 

Total 

20,000 

761 

3.81% 

Note:  Shaded  area  reflects  statistically  significant  differences. 
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Table  9:  Test  3  Comparison  of  Call  Responses  by  Income  and  Racial  Groups 


Exposure 

Mailings 

CaUs 

Response 

X  ^(p-value) 

African 

10,000 

296 

2.96% 

American 

White 

10,000 

465 

4.65% 

39:02  (p=(}.000) 

Low-Moderate 

10,000 

449 

4.49% 

25:64(p==0.000) 

Middle  -Upper 

10,000 

312 

3.12% 

African 

American 

Low-Moderate 

2,500 

177 

7.08% 

12>p8  ({p0,001) 

Middle-Upper 

2,500 

119 

4.76% 

White 

Low-Moderate 

2,500 

272 

10.88% 

‘  12.08  (p=0.001) 

Middle-Upper 

2,500 

193 

7.72% 

Low  -Moderate 

African 

2,500 

177 

7.08% 

American 

White 

2,500 

272 

10.88% 

rl4.80  (p^O.OOO) 

Middle-Upper 

African 

2,500 

119 

4.76% 

American 

White 

2,500 

193 

7.72% 

;i8:72;(i0i:oo(}) 

Note:  Shaded  areas  reflect  statistically  significant  differences. 


Table  10:  Test  3  Comparison  of  Call  Responses  by  Treatment  within  Racial  and 
Income  Groups 


Exposure 

Mailings 

Calls 

Response 

X  ^(p-value) 

African  American 
Automated 

3.2% 

15.19’(p=0.000):iiv: 

Live 

136 

2.7% 

White 

Automated 

5000 

236 

4.72% 

0.1ll(p=0.74) 

Live 

5000 

229 

4.58% 

Low-Moderate 

Automated 

5000 

253 

5.06% 

7.58  (pm:006): 

Live 

5000 

196 

3.92% 

Middle-Upper 

Automated 

5000 

143 

2.86% 

Live 

5000 

169 

3.38% 

2.24  (p-0.135) 

Low-Moderate 

African  American 
Automated 

97 

3.88% 

1.69  (p=0.193) 

Live 

White 

Automated 

2500 

156 

6.24% 

Live 

2500 

116 

4.64% 

Middle-Upper 

African  American 
Automated 

2500 

63 

2.52% 

0.422  (p-0.52) 

Live 

2500 

56 

2.24% 

White 

Automated 

2500 

3.20% 

:  . 

Live 

2500 

113 

4.52% 

1  5;87(iH)il5)  ^  1 

Note:  Shaded  areas  reflect  statistically  significant  differences. 


Table  11:  Test  3  Comparison  of  Call  Responses  to  the  Automated  Information  System 


Exposure 

Mailings 

Calls 

Response 

(p-value) 

African-American 

5000 

160 

3.20% 

White 

5000 

236 

4.72% 

/f5.19#^,000) 

Low-Moderate 

5000 

253 

5.06% 

*3|.82lQp^d)00)  ' 

Middle-Upper 

5000 

143 

2.86% 

Low-Moderate 

African- 

2500 

97 

3.88% 

American 

White 

2500 

156 

6.24% 

Middle-Upper 

African- 

2500 

63 

2.52% 

American 

White 

2500 

80 

3.20% 

2.08  (p=0.149) 

African-American 

Low-Moderate 

2500 

97 

3.88% 

:;«7.46ft^,006) 

Middle-Upper 

2500 

63 

2.02% 

White 

Low-Moderate 

2500 

156 

6.24% 

Middle-Upper 

2500 

80 

3.20% 

Note;  shaded  areas  reflect  statistically  significant  differences. 
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Table  12:  Test  3  Comparison  of  Call  Responses  to  the  Live  Counselor  Information 
System 


Exposure 

Mailings 

Calls 

Response 

x2  (p-value) 

African-American 

5000 

136 

2.72% 

White 

5000 

229 

4.58% 

24.59  (ij=0.0p6) 

Low-Moderate 

5000 

196 

3.92% 

2.07  (p=0.15) 

Middle-Upper 

5000 

169 

3.38% 

Low-Moderate 

African-A  merican 

2500 

80 

3.20% 

White 

2500 

116 

4.64% 

6.88  (^.00?)' 

Middle-Upper 

A frican-A  merican 

2500 

56 

2.24% 

White 

2500 

113 

4.52% 

j9:9(pM):ctoo)  • 

African-American 

Low-Moderate 

2500 

80 

3.20% 

35;(p^.to7):x: 

Middle-Upper 

2500 

56 

2.24% 

White 

Low-Moderate 

2500 

116 

4.64% 

0.04  (p=0.839) 

Middle-Upper 

2500 

113 

4.52% 

Note:  shaded  areas  reflect  statistically  significant  differences. 

In  addition  to  the  information  on  call  volume  patterns,  some  additional  results  were 
observed  from  the  Test  Three  research  effort.  These  include  caller  preference  related  to  time 
of  day,  specific  messages  accessed,  and  postcard  return  patterns. 

Timine  of  Calls:  As  in  Test  One,  in  Test  3  we  found  that  overall,  most  callers  preferred 
calling  in  the  evening,  between  6PM  and  12AM.  There  was  also  a  strongly  observed  split 
in  calling  time  preferences  along  racial  and  income  lines.  Along  income  lines,  lower 
moderates  showed  a  strong  preference  for  afternoon  calling  times,  while  middle  upper 
income  persons  called  mostly  in  the  evenings.  By  race,  Afiican  Americans  preferred  the 
afternoon,  while  Whites  preferred  evening.  All  racial  and  income  groups  showed  the  least 
preference  for  calling  during  the  night  (from  12AM  to  6AM). 
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Table  13  summarizes  the  high/low  call  periods  for  each  treatment  group. 


Table  13:  Test  3  High  and  Low  Calling  Periods 


Strateev 

Ethnicitv 

Income 

Level 

Call  Periods 

AALM 

AAM 

U 

WLM 

WMU 

M 

w 

LM 

MU 

Overall 

Call  Volume  Peak  - 
Overall 

Aft. 

Aft. 

im 

Eve. 

Aft. 

Eve. 

Aft. 

Eve. 

Eve. 

Lowest  Call  Volume- 
Overall 

Night 

Night 

Night 

Night 

Night 

Call  Volume  Peak-  Auto 

Aft. 

Aft. 

Eve. 

Aft. 

Eve. 

Aft. 

Eve. 

Aft. 

Call  Volume  Peak-  Live 

Aft. 

Mom. 

Eve. 

Eve. 

Aft. 

Eve. 

Eve. 

Eve. 

Eve. 

Lowest  C  Volume- Auto 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Lowest  C  Volume-Peak 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Night 

Definition  of  time  periods:  Morning  =  6am  -  12pm;  Afternoon  =12pm-6pm;  Evenings=6pm-12am; 
Night=12am-6am 

Strategy  Abbreviations:  AA=  African  American  Census  Area;  W=White  Census  Area;  LM=low  to  moderate 
income  group;  MU=Middle  to  upper  income  group 

ANECDOTAL  TESTS:  TESTS  TWO  AND  FOUR 

Test  Two  represents  a  brief  and  small  scale  assessment  of  the  impact  of  current 
technologies  in  enhancing  information  access.  This  assessment  attempted  to  obtain 
reactions  to  accessing  and  receiving  health  information  on-line  by  low  income  African 
Americans.  This  test  required  users  to  critique  an  on-line  bulletin  board  which 
computerized  the  library  of  the  Breast  Health  Information  Program.  Although  only  a 
small  number  of  subjects  were  included  in  this  test,  it  was  extremely  important  because  of 
the  growing  and  changing  use  of  technology  in  providing  information.  The  research 
design  and  sampling  size  for  Test  Two  were  not  established  to  provide  statistically 
significan  results.  However,  the  outcome  of  this  test  yielded  valuable  information 
indicators  about  the  use  of  on-line  health  information.  With  low-income  populations. 
Future  research  is  recommended  in  this  area. 

Test  Four  was  established  to  provide  anecdotal  information  and  greater  insight  related  to 
the  effectiveness  of  different  outreach  mediums  in  reaching  the  informationally  hard-to- 
reach.  This  test  used  a  much  larger  population  than  Test  Two  but  much  smaller  than  the 
sampling  size  needed  for  statistical  significance.  Like  Test  Two,  Test  Four  only  targeted 
low  income  African  Americans.  However,  this  test  compared  response  patterns  based  on 
different  outreach  mediums:  radio,  community  postings  and  mailings.  The  details  of 
Test  Four  provide  informative  insights  and  recommendations  for  further  studies  on 
outreach  strategies  for  the  informationally  hard-to-reach. 
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TEST  TWO 


Test  Two  was  a  small  evaluation  conducted  in  the  Annapolis,  Maryland  area  in  the 
Summer  of  1996  (in  between  Tests  One  and  Three).  The  purpose  of  Test  Two  was  to  test 
whether  or  not  accessing  breast  health  information  via  another  technology  medium  -  an 
“on-line”  computer  systenl  was  an  effective  complementary  automated  method  to  get 
information  to  low  income  populations.  The  goal  was  to  compare  use  of  the  two  systems 
to  determine  if  there  was  a  preferred  technology  mode  for  getting  health  information  to 
low  income  populations. 

To  test  this  strategy,  two  different  demonstration  systems  were  set  up  in  two  different 
low-income  housing  communities,  a  computer-based  information  system  and  an 
automated  telephone  information  system.  Response  rates  to  both  systems  were  tracked 
and  compared. 

Before  the  beginning  of  the  test,  women  from  three  separate  housing  communities  were 
invited  to  three  separate  receptions  to  introduce  the  test.  One  reception  group  was  told 
about  the  telephone  system,  one  was  told  about  the  computer  system,  and  one  was  a 
control  group. 

As  “incentive”  for  these  women  to  participate  in  testing  the  systems,  we  offered  them 
participation  in  a  contest  if  they  agreed  to  participate  in  a  follow-up  survey.  The  contest 
winner  would  receive  an  overnight  stay  at  a  local  hotel  and  dinner  for  two. 

Eighteen  women  agreed  to  participate  in  the  follow-up  survey.  Results  of  this  survey  are 
included  in  Appendix  F. 

After  the  follow-up  surveys  were  completed,  the  contest  winner  was  drawn  and  a 
certificate  was  presented. 

The  “computer  test”  consisted  of  a  computer  bulletin  board  system  with  message  content 
similar  to  the  telephone  information  system,  except  that  users  accessed  the  information 
“on-line”  through  a  computer,  rather  than  from  a  touch-tone  telephone.  The  computer 
bulletin  board  system  had  an  easy-to-use  menu  structure  that  only  required  mouse  clicks 
on  menu  choices  to  access  the  information. 

The  "telephone"  test  consisted  of  the  same  automated  telephone  information  system  used 
for  Tests  One  and  Three. 

The  Computer  Test 

The  computer  test  ran  from  June  6, 1996  through  July  18, 1996.  From  June  6  through 
July  2,  the  computer  was  situated  in  the  computer  room  at  Harbour  House,  one  of  the 
low-income  housing  communities.  Because  of  low  traffic  in  the  computer  room  and  little 
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use  of  the  system,  on  July  3  the  computer  system  was  moved  to  the  office  of  a  local 
family  practice  physician  where  more  people  would  have  access  to  it. 

We  were  able  to  track  the  following  data  about  users  of  the  system: 

•  Date,  time,  and  number  of  log-ons  to  the  computer  system 

•  Survey  Poll  Results 

•  Personal  Profile  Poll  Results 

•  “Improve”  Questionnaire  Results 


Date,  Time,  and  Number  of  Log-ons  to  the  Computer  System 

While  the  computer  system  was  situated  in  the  computer  room  in  the  low  income  housing 
complex,  no  hits  were  recorded,  indicating  that  the  system  was  not  used.  After  the 
system  was  moved  to  the  physician's  office,  during  the  two  weeks  spanning  July  3  to  July 
18,  30  hits  were  recorded. 

Nineteen  different  persons  logged  onto  the  system,  with  1 1  persons  logging  on  one  time 
each,  6  persons  logging  in  two  times  each,  and  2  persons  logging  in  three  times  each. 

Seven  hits  were  recorded  between  the  hours  of  9  to  10am.  Three  hits  took  place  between 
10am  and  12  noon.  The  majority  of  hits,  20,  took  place  between  the  hours  of  1pm  and 
6pm. 

On-Line  Survey  Poll  Results 

To  determine  a  “profile”  of  users  of  the  computer  system,  the  survey  poll  asked  users  four 
questions. 

1 .  Are  you  male  or  female? 

t 

There  were  26  responses  to  this  question.  1 1 .5%  (3  respondents)  were  male.  88.4%  (23 
respondents)  were  female. 

2.  How  old  are  you? 

There  were  26  responses  to  this  question.  42.3%  (1 1)  were  between  20  and  39  years  old; 
50%  (13)  were  between  40  and  59  years  old;  7.6%  (2)  were  age  60  or  older. 

3 .  What  is  your  race? 


There  were  20  responses  to  this  question.  70%  (14)  were  African  American;  5%  (1)  were 
Asian;  25%  (5)  were  Caucasian;  none  were  Hispanic;  none  were  Native  American;  and 
none  were  “other”. 
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4.  What  is  your  annual  household  income  (optional  question)? 

There  were  20  responses  to  this  question.  10%  (2)  were  below  $15,000;  10%  (2)  were 
between  $15,000  and  $24,999;  10%  (2)  were  between  $25,000  and  $34,999;  20%  (4) 
were  between  $35,000  and  $49,999;  15%  (3)  were  between  $50,000  and  $99,999;  5%  (1) 
were  over  $100,000;  and  30%  (6)  preferred  not  to  answer. 

Based  on  responses  to  these  questions,  most  users  of  the  system  were: 

•  Female 

•  Anywhere  between  the  ages  of  20  and  59  years  old 

•  African  American 

•  With  household  incomes  of  $35,000  to  $49,999 


On-Line  Personal  Profile  Poll  Results 

Respondents  were  asked  a  series  of  4  questions  in  an  effort  to  help  them  determine  which 
messages  would  be  most  helpful  to  them. 

1 .  Are  you  male  or  female? 

There  were  10  respondents  to  this  question.  90%  (9)  were  female;  10%  (1)  were  male. 

2.  Is  there  a  history  of  breast  cancer  in  your  immediate  family? 

There  were  10  respondents  to  this  question.  10%  (1)  had  a  family  history  of  breast 
cancer;  90%  (9)  did  not. 

3.  Have  you  had  your  breasts  examined  by  a  doctor  or  nurse  in  the  past  year? 

There  were  10  respondents  to  this  question.  50%  (5)  had  their  breasts  examined  in  the 
past  year;  50%  (5)  had  not. 

4.  Have  you  had  or  do  you  think  you  have  breast  cancer? 

There  were  10  respondents  to.this  question.  100%  (10)  had  not  had  or  did  not  think  they 
had  breast  cancer. 

Based  on  results  of  these  questions,  most  respondents: 

•  Were  female 

•  Did  not  have  a  family  history  of  breast  cancer 

•  Had  not  had  or  think  they  had  breast  cancer 
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Respondents  were  divided  evenly  on  having  a  breast  exam  in  the  past  year. 


On-Line  “Improve”  Questionnaire  Results 

Respondents  were  asked  a  series  of  six  questions  on  how  the  service  could  be  improved. 
Only  one  person  responded  to  this  questionnaire.  The  respondent  answered  as  follows: 

•  Had  already  completed  the  personal  profile 

•  Was  between  age  40  and  59  years  old 

•  Heard  about  the  service  through  “other” 

•  After  reading  the  library,  would  take  follow-up  action  by  making  an  appointment  with 
a  doctor  for  a  breast  exam 

•  Left  her  name  and  address  for  a  follow-up  survey  and  free  gift 
Telephone  Test 

The  telephone  test  period  ran  from  June  6, 1996  to  July  18, 1996.  During  the  test,  54 
calls  were  received.  Of  these  callers,  50  hung  up  at  the  “welcome”  message,  and  one 
caller  transferred  to  the  operator.  Only  three  callers  “pressed  one”  to  continue  using  the 
automated  telephone  system. 

The  majority  of  calls  occurred  on  Tuesday  evening  (20  calls)  and  Friday  evening  (20 
calls),  and  may  be  suspect  of  “hacker”  calls  according  to  patterns  previously  observed  by 
TeleSonic. 

Of  the  three  callers  who  pressed  one,  two  went  on  to  use  the  system.  These  two  callers 
listened  to  a  total  of  two  messages.  Neither  caller  ordered  a  fax  or  requested  that 
information  be  mailed  to  them. 

Of  the  two  callers,  one  participated  in  the  voluntary  survey.  This  respondent  provided  the 
following  answers  to  the  survey  questions: 

•  Between  ages  20  and  39 

•  Heard  about  this  service  through  a  friend 

•  After  listening  to  message  library,  would  take  follow-up  action  of  performing  a  breast 
self-exam 

Although  Test  Two  gave  us  only  anecdotal  information  and  no  statistically  significant 
inferences  can  be  drawn,  low  usage  of  the  automated  telephone  system  and  the  computer 
system  indicates  that,  used  alone,  they  may  not  be  the  most  effective  means  of 
disseminating  health  information  to  low  income  populations.  These  technologies  may 
better  serve  low-income  populations  when  used  in  conjunction  with  other  types  of 
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outreach,  such  as  one-on-one  counseling  or  direct  mail.  Results  from  Tests  One  and 
Three,  as  well  as  the  focus' group  studies,  clearly  indicate  that  when  the  automated  system 
is  used  in  conjxmction  with  another  form  of  promotion,  such  as  direct  mail,  that  this  is  a 
very  successful  means  of  getting  women  to  take  preventive  actions  against  breast  cancer. 

Literacy  may  have  had  an  additional  impact  on  the  willingness  of  these  low-income 
women  to  use  computer-based  information  systems.  This  further  supports  the 
effectiveness  of  using  an  automated  telephone  system,  where  literacy  is  not  a  necessary 
criteria  in  order  to  obtain  the  information  provided. 

TEST  FOUR 

Because  the  results  of  Tests  One  and  Three  supported  the  major  hypothesis,  a  fourth  test 
was  designed  to  test  two  additional  areas:  (1)  the  effectiveness  of  different  forms  of 
promotion  on  informationally  hard-to-reach  populations;  and  (2)  the  impact  of  "culturally 
sensitive"  messages  on  caller  response  rates.  The  fourth  test  targeted  the  AALM  target 
group  only  (TeleSonic's  informationally  hard-to-reach  population)  and  tested  caller 
response  rates  to  the  automated  telephone  information  system. 

Between  Tests  Three  and  Four,  the  automated  telephone  information  system  was  refined 
and  re-organized  to  provide  easier  and  quicker  menu  selections,  as  well  as  to  encourage 
callers  to  make  immediate  appointments  for  low  cost  mammograms  and  medical  services. 

In  Test  Four,  the  northeast  section  of  Baltimore  was  selected  for  its  high  concentration  of 
low-income  African  Americans,  as  well  as  its  proximity  to  Johns  Hopkins  University 
Cancer  Center.  Johns  Hopkins  University  Cancer  Center  offers  low-cost  mammograms 
and  medical  screening  appointments,  as  well  as  breast  cancer  prevention  classes. 

The  test  promotional  strategy  encompassed  direct  mail,  supermarket  flyer  distribution, 
and  radio  public  service  announcements.  In  addition,  two  different  promotional  messages 
were  tested  in  each  of  the  three  promotional  vehicles:  a  "generic"  message  (untreated)  and 
a  "culturally  sensitive"  message  (treated). 

For  the  direct  mail  component,  5,000  postcards  were  mailed  to  an  African  American  low- 
income  target  group  in  Baltimore,  Maryland.  Twenty-five  hundred  cards  were  generic, 
and  2,500  cards  had  a  "culturally  sensitive"  sticker  applied  to  them. 

For  the  supermarket  component,  a  total  of  5,000  postcards  were  distributed  to  three 
SuperPride  supermarkets.  The  stores  were  selected  for  their  geographic  composition  and 
proximity  to  Johns  Hopkins  University  Hospital,  where  the  low  cost  mammograms, 
doctor  appointments,  and  educational  classes  were  being  offered.  In  addition  to  strategic 
placement  at  the  courtesy  counters,  posters  advertising  the  card  giveaway  were  placed  in 
each  store.  Sample  brochures  are  included  in  Appendix  G. 
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For  the  radio  component,  two  public  service  armouncements  (PSAs)  were  created,  a 
generic  PSA  and  a  culturally  sensitive  PSA.  PSAs  were  aired  on  two  different  radio 
stations  with  strong  listenership  in  our  target  group.  A  copy  of  the  radio  public  service 
announcements  is  included  in  Appendix  H. 


Table  14:  Test  Four  Design  (Direct  Mail,  Supermarket  Flyers,  and  Radio  PSAs) 


TARGET 

GROUP 

NUMBER  OF 

AUTOMATED 

TELEPHONE 

INFORMATION 

SYSTEM 

POSTCARDS 

MAILED 

NUMBER  OF 
FLYERS 

DISTRIBUTED  AT 
SUPERMARKETS 

NUMBER  OF  RADIO 
STATION  PUBLIC 
SERVICE 

ANNOUNCEMENTS 

(PSAs) 

AALM 

2,500 

2,500 

41 

Untreated 

AALM  Treated 

2,500 

2,500 

45 

TOTALS 

5,000 

5,000 

86 

Test  Four  was  conducted  in  the  Baltimore  metropolitan  area  in  the  Spring  of  1997.  The 
original  purpose  of  the  test  would  be  to  take  a  geographic  area  that  had  already  been 
tested  (Baltimore  —  Test  One)  and  whose  results  had  been  validated  by  another 
geographic  area  (Philadelphia  --  Test  Three),  and  add  an  additional  “medium”  such  as 
radio  or  newspaper  to  the  same  test  design  to  determine  its  impact. 

However,  Test  Four  was  repositioned  to  address  some  additional  points  raised  by 
proposal  evaluators  and  Project  Advisors,  and  to  strengthen  the  available  useful  data  on 
the  automated  system  as  an  effective  means  of  disseminating  health-related  information. 
The  location  (Baltimore,  MD)  and  the  timing  of  the  test  (Spring  1997)  did  not  change. 

As  such,  the  goals  of  Test  Four  were  to: 

•  Test  the  impact  of  alternate  media  (to  the  direct  mail  approach  used  in  earlier  tests)  on 
call  volume  to  the  automated  telephone  information  system 

•  Test  the  impact  of  cultural  sensitivity  in  media  messages  on  call  volume  into  the 
automated  telephone  information  system 

•  Increase  the  opportunities  for  callers  to  initiate  Level  Two  and  Three  behaviors 
(increasing  commitments  to  healthy  behavior)  directly  through  the  automated 
telephone  system 

•  Decrease  the  caller  hang-up  rate 
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Increase  caller  time  on  line 

Increase  the  number  of  calls  into  the  system 


In  addition  to  the  goals  outlined  above,  several  issues  cited  by  proposal  evaluators  and 
Project  Advisors  were  also  addressed  as  part  of  Test  Four.  Industry  statistics  were 
researched  and  compared  to  the  results  achieved  for  the  Breast  Health  Information  system 
in  the  following  areas: 

•  Industry  hang-up  rates  for  automated  telephone  information  systems  (as  a  comparison 
to  hang-up  rates  observed  in  our  automated  system) 

•  “Hold”  times  for  live  counselor  systems  (as  a  comparison  to  “time  on  line”  of  the 
automated  system) 

•  Cancer  Information  Service  (CIS)  abandon  call  rates  (as  a  comparison  to  hang-up 
rates  observed  in  the  automated  system) 

•  CIS  average  length  of  call  (as  a  comparison  to  the  average  length  of  call  observed  in 
the  automated  system) 

•  Average  direct  mail  industry  response  rates  (as  a  comparison  to  the  direct  mail 
response  rates  achieved  in'  our  tests) 

To  accomplish  the  goals  outlined  for  Test  Four,  as  well  as  to  effectively  address  the 
remaining  project  issues.  Test  Four  was  conducted  in  the  Baltimore  metropolitan  area  in 
the  Spring  of  1997,  and  ran  for  approximately  seven  weeks.  Although  the  test  did  not 
provide  statistically  significant  results,  it  allowed  us  to  measure  several  important  caller 
patterns,  preferences,  and  behaviors. 

The  test  focused  on  three  main  areas: 

•  Testing  the  impact  of  alternate  media  (to  direct  mail)  on  call  volume  into  the 
automated  telephone  information  system 

•  Testing  the  impact  of  cultural  sensitivity  in  media  messages  on  call  volume  into  the 
automated  telephone  information  system 

•  Increasing  the  opportunities  for  callers  to  initiate  Level  Two  and  Three  behaviors 
directly  through  the  automated  telephone  system 

Test  Four  sought  to  find  ways  to  increase  call  volume  into  the  automated  telephone 
system,  rather  than  compare  caller  preferences  for  an  automated  versus  a  live  counselor 
system  (already  tested  and  proven  in  Tests  One  and  Three).  Therefore,  the  live  counselor 
system  was  not  included. 

Also,  Test  Four  sought  to  measure  the  effectiveness  of  the  automated  telephone  system  as 
a  means  of  reaching  “informationally  hard  to  reach”  populations.  The  test  targeted 
African  American  populations;  however,  white  populations  were  not  targeted,  and 
income  level  criteria  were  not  tested. 
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Approach  for  Measuring  the  Impact  of  Different  Media  on  Call  Volume 

To  measure  the  impact  of  different  media  as  a  means  of  generating  call  volume  into 
the  automated  telephone  information  system,  three  different  media  were  tested; 

•  Direct  mail 

•  Flyers  placed  in  grocery  stores 

•  Public  service  radio  announcements 

All  media  promoted  the  automated  telephone  information  system  as  a  method  of 
obtaining  free,  confidential  information  on  breast  health. 


Approach  for  Measuring  the  Impact  of  Culturally  Sensitive  Media  Messages  on  Call 
Volume 

To  measure  the  impact  of  cultural  sensitivity  in  media  messages  on  call  volume,  each 
of  the  three  media  had  two  test  groups  (for  a  total  of  six  test  groups): 

•  A  “treated”  group,  that  received  a  culturally  sensitive  media  message  and  a  culturally 
sensitive  audio  message  on  the  automated  system 

•  An  “untreated”  group,  that  received  a  generic  (non-culturally  sensitive)  media 
message  and  generic  audio  messages  on  the  automated  system 


Approach  for  Increasing  Level  Two  and  Three  Behaviors 

To  give  callers  the  opportunity  to  initiate  Level  Two  and  Three  behaviors,  a  new  call 
flow  was  developed  with  an  “action  script”,  that  encouraged  callers  to  take  immediate 
action  in  a  variety  of  preventive  care  areas.  All  six  test  groups  heard  the  same  action 

script.  “Action  opportunities”  included: 

: 

•  Immediate  sign-up  for  a  free  mammogram 

•  Immediate  sign-up  for  free  classes  on  breast  health  at  an  area  hospital 

•  Immediate  transfer  to  sign  up  for  a  low  cost  doctor  appointment 

•  Immediate  request  for  information  by  mail  or  fax 

•  Immediate  transfer  to  a  live  counselor 

•  Immediate  ability  to  leave  a  message  with  questions  and  comments 

Design  Approach 

Direct  Mail.  5,000  postcards  were  sent  via  direct  mail  to  African  American  households. 
2,500  of  these  postcards  had  a  culturally  sensitive  message;  2,500  postcards  were  generic. 
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Retail  Establishments.  5,000  postcards  were  placed  in  three  SuperPride  grocery  stores 
frequented  by  lower  moderate  income  level  African  Americans.  2,500  postcards 
contained  a  culturally  sensitive  message;  2,500  were  generic. 

Public  Service  Radio.  Two  African  American-specific  radio  stations  (WBRG  and 
WCAO)  in  Baltimore  were  targeted  and  selected.  A  culturally  sensitive  public  service 
announcement  was  played  in  half  the  radio  spots;  a  generic  public  service  announcement 
was  played  in  the  other  half  of  the  radio  spots. 

Each  of  the  six  test  groups  had  a  separate  800  number.  Measurements  and  comparisons 
were  made  between  total  call  volume  generated  by  the  three  different  media,  total  call 
volume  between  culturally  sensitive  and  generic  media  messages,  and  individual  call 
volumes  by  media  and  by  message. 

We  also  measured  the  effectiveness  and  popularity  of  the  caller  “action  opportunities”, 
and  compared  the  results  to  the  type  of  media  that  generated  the  call. 


Test  Four  Results 

Although  Test  Four  did  not  yield  statistically  significant  results,  the  promotional  methods 
tested  and  the  caller  behaviors  observed  provided  us  with  some  very  useful  anecdotal 
information. 

As  detailed  earlier  in  this  section.  Test  Four  compared  caller  response  rates  to  direct  mail, 
supermarket  flyers,  and  radio  public  service  announcements  (PSAs)  as  a  means  of 
promoting  the  automated  telephone  information  system.  In  addition,  two  different 
promotional  messages  were  tested:  a  generic  message  (untreated)  and  a  culturally 
sensitive  message  (treated). 

For  the  seven  week  test  period,  a  total  of  377  calls  were  received.  Of  these  calls,  208 
were  from  "imtreated"  strategies",  while  169  were  from  "treated"  strategies.  The  call 
counts  within  the  individual  strategies  are  detailed  in  the  table  below. 
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Table  15:  Test  Four  Call  Counts  by  Strategy 


1  PROMOTIONAL  METHOD 

NUMBER  OF  PLACEMENTS 

CALL  COUNT 

Mail  -  Untreated 

2,500  cards 

48 

Mail  -  Treated 

2,500  cards 

79 

Supermarket  -  Untreated 

1,300  cards* 

83 

Supermarket  -  Treated 

500  cards* 

37 

Radio  -  Untreated 

24  PSAs** 

77 

Radio  -  Treated 

28  PSAs** 

53 

TOTAL 

N/A*** 

377 

*  A  total  of  2,500  untreated  and  2,500  treated  cards  were  provided  to  three  different 

supermarkets.  Figures  noted  represent  the  number  of  cards  that  were  actually  picked  up 
by  a  shopper. 

**  Two  different  radio  stations  played  both  the  untreated  and  the  treated  PSAs.  One 
station  alternated  the  play  schedule  between  the  two  messages  by  week,  while  the  other 
station  alternated  the  play  schedule  by  message. 

***  Because  of  the  differing  nature  of  the  placements,  a  total  cannot  be  calculated. 

As  detailed  in  the  table  below,  a  comparison  of  treated  versus  untreated  response  rates 
shows  that  the  untreated  (generic)  messages  produced  a  greater  overall  caller  response 
rate.  In  the  case  of  the  radio  spots,  the  untreated  message  received  a  higher  response  rate, 
while  in  the  direct  mail  strategies,  the  treated  cards  received  a  higher  response.  In  the 
case  of  the  supermarkets,  the  fact  that  a  larger  number  of  untreated  cards  were  picked  up 
than  treated  cards  may  have  had  an  impact  on  the  higher  response  rate  to  the  untreated 
message. 
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Table  16:  Test  Four  Call  Counts  by  Treatment 


UNTREATED 

Mail 

48 

Supermarket 

83 

Radio 

77 

Total  Untreated 

208 

TREATED 

Mail 

79 

Supermarket 

37 

Radio 

53 

Total  Treated 

169 

The  table  below  details  the  caller  response  rate  by  promotional  method.  Surprisingly,  all 
three  methods  had  a  very  similar  overall  caller  response  rate,  despite  the  fact  that  different 
numbers  of  cards  or  radio  spots  were  placed.  Because  the  placement  mechanisms  were  so 
different,  it  is  difficult  to  draw  comparisons  between  groups.  The  intent  of  Test  Four  was 
to  draw  comparisons  within  groups. 

Table  17:  Test  Four  Call  Count  by  Promotional  Method 


Mail 

127 

Supermarket 

120 

Radio 

130 

Total  Calls 

377 

Test  Four  also  provided  some  valuable  insight  into  the  behavior  of  callers  to  the 
automated  system.  Of  the  three  forms  of  promotion,  the  respondents  from  the  direct  mail 
campaign  and  the  supermarket  displays  were  fairly  "active"  users  of  the  automated 
system  and  the  information  provided,  while  respondents  from  the  radio  spots  did  not  take 
advantage  of  much  of  the  information  provided.  Although  there  were  130  radio  spot 
respondents  (the  highest  of  the  three  forms  of  promotion),  none  of  the  callers  went  past 
the  main  menu.  In  contrast,  respondents  to  the  direct  mail  and  supermarket  cards  were 
very  active  users  of  the  information.  These  callers  utilized  the  message  library,  the  breast 
cancer  prevention  steps,  the  transfer  to  a  live  counselor  option,  as  well  as  the  optional 
caller  survey. 

One  possible  explanation  for  the  stark  contrast  between  the  direct  mail/supermarket 
respondents  and  radio  respondents  may  be  that  both  the  direct  mail  and  the  supermarket 
respondents  received  a  visual,  tangible  card  that  explained  the  service  and  prepared  them 
for  reaching  an  automated  telephone  information  system.  The  radio  respondents,  on  the 
other  hand,  had  only  the  audio  message  to  go  on,  and  may  have  been  somewhat  surprised 
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when  they  reached  an  automated  system,  rather  than  a  live  person.  The  message  that  the 
"breast  health  information  line"  was  actually  an  automated  system  may  not  have  been  as 
readily  apparent  to  radio  listeners  as  we  originally  thought. 

Within  each  promotional  method,  there  were  no  clear  behavior  differences  between 
untreated  and  treated  strategies.  In  some  areas  of  the  system,  "treated"  callers  were  more 
"active"  users  of  information,  while  in  other  areas  the  "untreated"  callers  were  more 
active  users. 

The  table  below  details  some  of  the  caller  behaviors  by  promotional  method  and 
treatment. 


Table  18:  Test  Four  Caller  Actions  by  Strategy 


Caller 

Action 

#  of  Mail 
Untreatd 
Callers 

#of 

Mail 

Treatd. 

Callers 

#of 

Super¬ 

market 

Untreatd 

Callers 

#of 

Super¬ 

market 

Treatd. 

Callers 

#of 

Radio 

Untreatd 

Callers 

#of 

Radio 

Treatd. 

Callers 

Total  # 
of 

Callers 

Greeting 

48 

79 

83* 

37* 

77 

53 

Main  Menu 

34 

60 

60 

25 

38 

5 

Prevention 

Steps 

Menu 

3 

10 

7 

4 

0 

0 

24 

Message 

Library 

Menu 

18 

23 

14 

14 

0 

0 

69 

Live 

Counselor 

Transfer 

3 

4 

2 

0 

0 

0 

9 

Voluntary 

Caller 

Survey 

5 

17 

6 

3 

0 

0 

31 

TOTAL 

ACTIONS 

193 

115 

58 

732 

*  As  noted  previously  in  this  section  in  Table  15,  the  fact  that  a  larger  number  of 
untreated  cards  were  picked  up  than  treated  cards  may  have  had  an  impact  on  the  higher 
response  rate  to  the  untreated  message. 


In  siunmary,  the  anecdotal  results  of  Test  Four  indicate  that  there  are  no  clear  differences 
in  caller  response  rates  either  by  promotional  method  or  by  treatment.  Caller  response 
rates  by  promotional  method  (mail,  supermarket,  or  radio)  were  almost  even,  while 
response  rates  by  message  (treated  vs.  untreated)  varied  by  the  form  of  promotion  used. 
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We  did  find  that  direct  mail  respondents  were  the  most  active  users  of  the  information, 
supermarket  respondents  were  also  fairly  active  users,  but  radio  respondents  were  not 
active  users  of  the  information. 


B.  METHODS  AND  RESULTS  RELATIVE  TO  GOALS  OF  THE 
RESEARCH 

This  section  compares  each  of  the  nine  major  project  tasks  with  the  results  achieved.  As 
detailed  below,  all  nine  tasks  were  successfully  completed. 

TASK  1:  Determine  content  of  the  information  system 

a)  Establish  advisory  panel 

b)  Seek  input  from  advisors 

c)  Identify  appropriate  topics 

d)  Seek  expert  input  on  topics 

e)  Develop  final  list  of  topics 

f)  Determine  appropriate  message  titles 

All  activities  related  to  Task  1  were  successfully  accomplished. 

A  voluntary  advisory  pane!  was  established  that  included  a  broad-based  representation  from 
the  local  health  department,  health  educators,  cancer  patients,  cancer  research  and 
telecommunications  experts,  medical  practitioners  engaged  in  oncology  and  representatives 
of  breast  cancer  support  groups,  among  others.  A  listing  of  the  project  advisors  and 
consultants  is  included  in  Appendix  I.  These  advisors  provided  input  and  assistance 
throughout  the  project  in  areas  including  topic  selection,  message  content,  and  testing 
strategy. 

Refinements  to  the  research  design  evolved  in  the  early  stages  of  the  research  project, 
during  the  summer  and  fall  of  1994,  sparked  by  the  synergistic  effect  of  engaging  a  broad 
based  representation  of  project  advisors.  These  refinements  were  aimed  to  meet  two  basic 
goals: 

1 .  To  clearly  define  the  research  design  to  support  operational  considerations,  and 

2.  To  determine  if  refinement  was  needed  to  the  research  strategy  that  could  improve  the 
overall  results. 

The  refinements  enhanced  and  clearly  define  the  proposed  research  plan.  Three 
components  were  added  or  modified  as  a  result  of  deliberations: 

1 .  The  inclusion  of  a  pretest  as  a  precursor  to  the  demonstration  tests. 

2.  The  inclusion  of  a  process  for  input  and  recommendations  firom  the  target  groups 
through  the  use  of  focus  groups;  and 
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3.  Development  of  a  series  of  comparative  tests  to  look  at  the  difference  in  caller 
preference  for  automated  and  live  information  systems  across  ethnicity,  income  and 
geographical  areas.  The  test  was  expanded  to  include  comparative  analysis  of  the 
difference  in  response  patterns  based  on  a  change  in  the  outreach  strategy. 

More  than  70  topics  were  originally  researched  and  identified  by  project  staff  Then  after 
receiving  input  from  advisors  through  rankings  and  discussions,  the  number  of  topics  was 
reduced.  Because  differences  were  observed  in  information  interest  along  racial  and 
income  lines,  a  series  of  focus  groups  were  used  to  determine  which  topics  to  include  in  the 
automated  telephone  system. 

The  focus  groups  provided  invaluable  insight  and  information,  and  helped  identify  and 
clarify  issues  for  consideration  in  the  design  and  operation  of  the  information  system  and 
outreach  efforts.  Information  received  from  the  focus  groups  was  incorporated  throughout 
the  course  of  the  research  project. 

This  information  formed  the  basis  of  a  work  session  with  health  educators  and  an  oncology 
expert,  where  the  topics  were  reduced  to  a  final  15. 

The  results  of  the  two  to  three  month  period  of  development  and  deliberations  yielded  a 
total  of  25  messages  within  five  categories  as  noted  in  Table  19. 
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Table  19:  Message  Category  and  Titles  -  Automated  Telephone  System 


MESSAGE  CATEGORY 

MESSAGE  TITLES 

1.  Breast  Cancer:  What  You  Need  to  Know 

1.  What  is  breast  cancer? 

2.  Put  fear  aside  -  breast  cancer  is  curable. 

3.  Breast  cancer  myths. 

4.  Every  breast  lump  or  pain  is  not  cancer. 

5.  Breast  cancer  and  African  American  women 

6.  Men,  you  can  get  breast  cancer  too 

2.  Breast  Cancer  Risk  Factors 

7.  Are  you  at  risk  for  getting  breast  cancer? 

8.  Do  genetics  and  fffliy  history  play  ‘  .7r:;i 

9.  Breast  cancer^  the  pill  and  menopause  \  f 

10  Your  lifestyle  md  how  it  a^ects^breast  mcer;V  '.Sil 

1 1 .  How  to  lower  your  risk  for  getting  breast  cancer 

11  Role  of  Diet '  ^  ^  ^ .  j 

13.  Role  of  Exercise.^  \  Ci 

3,  Breast  Health 

14.  Breast  examination  (Related  information  by  fax  or  mail  also 
available) 

15,  AU  about  rnammo^^  (Rel&el^Solmafion  by  fax  or  "mfl 

also  available)''"'/,  /VN-  'M 

16.  When  should  you getamamtnbg?:am?f|^^$v  v^  ^  >|| 

17,  Mammogram  semce^; 

18.  Where  do  I  get  more  information  about  breast  cancer? 

4.  Treatment  Options 

19.  How  can  breast  cancer  be  treated? 

;20;Brea^'MFa«a:suri^ 

^21,  Chemotherapy,  radiation  treatment  and  Hormones  ! 

^22,  Medical  frontiers-new  research/ ■  'r^ 

5.  Recovery  and  Follow-up 

23.  There  is  a  life  after  breast  cancer. 

24.  Community  bulletin  board  (Related  information  by  fax  or  mail 
available) 

25.  How  should  I  support  a  friend  or  relative  with  breast  cancer? 

NOTE;  Shaded  messages  represent  sub-messages  that  are  not  listed  in  the  directory,  but  are  branched  in  the 
menu  stracture. 


TASK  2:  Research  technical  system  design  issues 

a)  Establish  minimum  technical  parameters 

b)  Investigate  add-on  technology  options 

c)  Determine  available  system  options 

d)  Determine  a  technical  configuration 

Technical  design  included  those  activities  associated  with  acquisition  and  installation  of  the 
hardware,  software  and  telephone  lines  to  support  the  telecommunication  system.  This 
effort  also  included  "loading"  the  system  with  the  message  content. 

The  telecommunication  systern  operated  on  a  personal  computer  platform,  using  Verbatim 
telecommunications  software.  In  selecting  the  software,  research  was  conducted  to  ensure 
that  the  system  that  would  provide  the  necessary  data  to  enable  TeleSonic  to  respond  to  the 
questions  posed  in  the  hypothesis. 

The  equipment,  software,  and  telephone  lines  were  configured  to  support  the  basic  call 
pattern  for  the  system.  Call  flows  were  developed  to  give  callers  options,  yet  to  capture  as 
much  data  as  possible.  Appendix  J  shows  the  flow  of  caller  activity  for  the  automated 
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system.  Appendix  K  shows  the  flow  of  caller  activity  for  those  calling  the  live  counselors 
at  the  Cancer  Information  Service  and  the  American  Cancer  Society. 


TASK  3:  Research  issues/topics  and  produce  messages  identified  for  the  information 
system’s  audiotex/faxiotex  library 

a)  Identify  and  secure  any’ additional  required  resources 

b)  Conduct  research 

c)  Write  new  scripts 

d)  Edit  scripts 

e)  Use  advisors/consultants  to  review  scripts 

f)  Identify  voice  production  talent 

g)  Produce  scripts 

h)  Install/dub  messages  onto  system 

All  activities  related  to  Task  3  were  successfully  completed. 

As  described  in  Task  1,  above,  more  than  70  topics  were  originally  identified.  Then  after 
receiving  input  from  advisors  through  rankings  and  discussions,  the  number  of  topics  was 
reduced.  This  information  formed  the  basis  of  a  work  session  with  health  educators  and  an 
oncology  expert,  where  the  topics  were  reduced  to  a  final  15.  Scripts  were  then  developed 
for  these  topics.  A  complete  set  of  scripts  is  included  in  Appendix  L. 

Using  research  information  and  advice  from  advisors,  scripts  were  written  and  edited  by 
project  staff.  Each  script  was  then  grouped  and  read  by  at  least  three  advisors.  All  scripts 
were  reviewed  by  an  oncology  expert.  The  input  from  these  two  external  sources  resulted 
in  a  re-editing  of  scripts  and  a  final  work  session  with  the  oncology  practitioner.  The  scripts 
were  then  professionally  recorded  for  the  telephone  information  system,  and  dubbed  onto 
the  system  by  project  staff. 

A  disclaimer  message  was  also  included  in  the  system,  which  stated  "Messages  and  titles  in 
this  library  are  the  property  of  TeleSonic.  The  information  you've  just  heard  is  for 
educational  purposes  only.  If  you  need  medical  advice,  please  contact  a  doctor  or 
appropriate  health  care  professional." 


TASK  4:  Acquire  and  set-up  the  demonstration  system  model. 

a)  Obtain  price  data 

b)  Select  best  equipment  options 

c)  Acquire  the  necessary  equipment 

d)  Set  up  equipment  for  testing  and  loading 
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e)  Install  the  technical  configuration  design 

f)  Beta  test  the  application  in-house 

g)  Make  final  modifications  to  the  technical  design 

h)  Install  system 

i)  Provide  training. 

All  activities  related  to  Task  4  were  successfully  accomplished. 

As  described  in  Task  2,  the  telecommunication  system  operated  on  a  personal  computer 
platform,  using  Verbatim  telecommunications  software.  In  selecting  the  software,  research 
was  conducted  to  ensure  that  the  system  that  would  provide  the  necessary  data  to  enable 
TeleSonic  to  respond  to  the  questions  posed  in  the  hypothesis. 

Once  the  equipment  was  acquired,  set  up,  and  tested,  the  application  was  loaded.  Then 
all  staff  participated  in  an  internal  beta  test  of  the  system.  After  modifications  were 
made,  an  additional  external  beta  test  was  conducted  encompassing  all  project  advisors. 

After  the  external  beta  test,  final  modifications  were  made  to  the  system.  Then,  all  staff 
were  trained  in  system  operation,  maintenance,  and  data  collection  procedures. 


TASK  5:  Establish  system  and  data  retrieval  routines. 

a)  Define  evaluation  criteria 

b)  Develop  guidelines  for  maintenance  and  monitoring 

c)  Identify  dynamic  information  sources 

d)  Designate  personnel  responsible  for  tracking  data. 

All  activities  related  to  Task  5  were  successfully  completed. 

The  technical  system  was  designed  to  collect  data  related  to  six  issues: 

1 .  Number  of  callers  firom  a  particular  area 

2.  Number  of  topics  accessed  by  a  caller,  general  preferences,  preferences  by 
census  area,  preferences  for  calls  taking  the  personal  assessment 

3.  Number  of  callers  who  transferred  to  a  live  counselor  by  census  area 

4.  Analysis  of  caller  assessments  with  caller  topical  preference,  including  live 
counselor  transfers,  by  call  census  area 

5.  Number  of  callers  requesting  fax  or  hard  copy  by  census  area 

6.  Number  of  callers  by  census  area  who  responded  to  follow-up  surveys 

In  preparation  for  the  activation  of  Test  One,  system  routines,  guidelines,  and  monitoring 
procedures  were  established.  In  addition,  staff  training  was  conducted  covering  procedures 
for  handling  operator  calls  and  daily  maintenance  of  the  system.  The  system  was  monitored 
and  data  was  collected  on  a  regular  basis. 
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Data  on  the  call  behavior  by  treatment  group  was  collected  automatically  by  a  call  log.  It 
should  be  noted  that  the  originating  phone  number  of  the  call  was  not  tracked,  to  protect 
the  identity  of  callers.  The  only  identifying  information  came  from  those  callers  who 
voluntarily  offered  to  provide  follow-on  feedback.  Computer  program  routines  were 
written  to  purge  data  and  generate  reports  on  variables  needed  for  research  analysis.  Data 
were  purged  to  exclude  staff  calls  for  daily  line  qualification,  review  and  system 
evaluation.  These  calls  were  coded  for  easy  extraction.  In  addition,  computerized  calls 
into  the  system  were  purged  based  on  computer  programming  patterns  that  were 
discerned.  These  included  set  patterns  of  seven  calls  to  the  same  telephone  number  at  six 
to  seven  second  intervals.  Call  reports  were  also  monitored  to  ensure  that  no  calls  were 

received  from  areas  blocked  from  the  call  region. 

} 

Significant  amounts  of  data  were  available  to  us  through  the  software;  however,  report 
generation  was  labor  intensive  and  time  consuming.  To  improve  our  data  reporting 
capabilities,  we  developed  a  database  report  generator  package  that  allowed  us  to  run  ad  hoc 
reports  for  any  time  period  specified  by  the  end  user.  Reports  could  be  run  for  as  little  as  a 
day,  or  as  long  as  three  months.  These  ad  hoc  reports  were  available  simply  by  using  “point 
and  click”  technology.  The  following  reports  were  available  in  the  report  generator 
package: 

•  Message  and  Document  Report  (mail  and  fax) 

•  Call  Volume  Report 

•  Survey  Report 

•  Profile  Report 

These  reports  are  included  in  Appendix  D,  Test  One  Data  Reports  and  Appendix  E,  Test 
Three  Data  Reports. 

TASK  6:  Develop  and  implement  the  outreach  and  promotional  campaign  plans  for 
the  two  target  groups  within  each  region. 

a)  Define  the  target  audience 

b)  Investigate  impact  of  progrzim  name 

c)  Develop,  produce,  and  disseminate  promo  materials 

d)  Enlist  the  assistance  of  professional  and  community  networks 

e)  Conduct  outreach  and  promo  efforts. 

To  test  its  hypothesis,  TeleSonic  identified  four  main  target  groups  by  race  and  income 
level:  a  White  Low-moderate  income  group  (WLM),  a  White  Middle-Upper  income 
group  (WMU),  an  African  American  Low-moderate  income  group  (AALM),  and  an 
African  American  Middle-Upper  group  (AAMU).  This  structure  allowed  TeleSonic  to 
measure  response  rates  to  a  direct  mail  campaign  by  both  race  and  income. 

In  developing  the  program  name,  assistance  was  enlisted  from  project  advisors. 
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Two  different  focus  groups  were  conducted  during  the  first  phase  of  the  test  in  developing 
the  promotional  campaign.  The  first  series  of  four  focus  groups  were  used  to  generate 
ideas,  recommendations  and  reactions  used  in  the  development  of  the  direct  mail  campaign. 
These  groups  were  identified  by  organizations  in  the  target  census  areas  that  have 
memberships  representative  of  the  test  design.  One  focus  group  was  identified  for  each  of 
the  four  targeted  population  groups  for  the  study.  The  only  modification  was  that  the  focus 
groups  for  the  White  low  to  moderate  income  group  became  a  mixed  group  of  white  and 
Afiican  American  females. 

Some  of  the  key  findings  are  noted  in  the  following  tables.  In  most  cases,  the  information 
summarized  in  the  tables  is  based  on  the  most  frequently  referenced  response. 

1 .  Why  did  you  offer  to  participate  in  a  discussion  on  breast  cancer? 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1 .  Personal  concerns  about  breast  cancer 

X 

X 

X 

na 

2.Family  history 

X 

X 

X 

na 

3.  Lifestyle  concerns 

X 

X 

na 

The  responses  to  this  question  were  fairly  consistent  across  each  of  the  focus 
groups.  Most  frequently,  the  members  stated  a  relative  with  breast  cancer  or  their 
own  concern  about  the  disease  as  their  primary  reason  for  participation.  Lifestyle 
comments  included  statements  like  busy  schedules  and  personal  health  matters. 
This  question  was  not  asked  of  the  White  Upper  to  middle  income  group. 
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2.  What  is  your  attitude  toward  breast  cancer? 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1.  Fear  or  "I'm  scared" 

X 

X 

X 

X 

2.  Don't  want  to  know 

X 

X 

X 

3.  Resentment  toward  the  medical 
establishment  for  not  supporting  standards 
they  promulgate 

X 

4.  Believe  in  preventive  measures  or  want 
to  know  more 

X 

X 

X 

The  range  of  responses  to  this  question  was  broader.  Most  frequently  referenced  across 
all  groups  was  the  issue  of  fear,  confiision  or  feeling  of  being  scared.  Also  included 
were  comments  describing  actions  that  clearly  showed  an  interest  in  taking  preventive 
actions  like  getting  a  mammogram,  self-examinations,  or  wanting  to  know  more  about 
the  topic. 

3.  Why  do  you  suppose  breast  cancer  deaths  have  risen  18%  in  the  last  20  years  in  African 
American  women  of  all  ages? 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1 .  Lack  of  preventative  measures 

X 

X 

X 

na 

2.  Other  priorities 

X 

X 

na 

3.  Requires  extra  cost,  efforts  or  insurance 

X 

X 

X 

na 

4.  Fear 

X 

X 

na 

The  responses  summarized  for  this  question  consistently  related  to  the  issue  of 
requiring  additional  cost,  effort,  or  insurance.  All  three  were  mentioned  by  the  low 
to  moderate  African  American  income  group,  with  cost  being  the  most  frequently 
referenced  problem  for  the  low  to  moderate  mixed  race  group.  No  response  is 
provided  from  the  upper  to  middle  income  white  focus  group.  Their  general 
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response  to  this  question  was  that  it  is  possible  that  the  death  rate  is  higher  because 
there  is  a  better  reporting  system.  They  also  voiced  concerns  about  chemicals  in 
the  environment  and  the  local  incidence  pattern. 

4.  Death  related  to  cancer  can  be  greatly  reduced  through  eating  right,  not  smoking,  and 
finding  cancer  early.  What  would  be  the  best  thing  to  say  to  women  to  get  them  to  do  these 
things? 


Response  Type 

: 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

l.More  advertisement 

X 

2.Push  benefits/hope  with  early  treatment, 
promote  good  news 

X 

X 

X 

X 

3. Use  scare  as  a  motivator 

X 

4  Show  empowerment  through  information 

X 

X 

While  a  few  women  in  the  mixed  low  to  moderate  income  group  were  motivated 
by  scare  tactics,  all  women  wanted  to  hear  positive,  information  about  possibilities 
and  success  stories  and  empowerment.  One  group  even  proposed  the  idea  about 
using  well  known  stores  like  Nordstrom  to  provide  services  for  persons  who  have 
had  their  breasts  removed.  Only  one  group,  the  low  to  moderate  income  Afiican 
American  group,  indicated  the  importance  of  enhanced  advertisement  similar  to  the 
ads  on  smoking.  Discussions  from  all  groups  inferred  advertisement  as  being 
necessary. 
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5,  Which  is  better  to  tell  people  about: 

a.  Facts  about  breast  cancer  deaths 

b.  Ways  to  help  lessen  the  risk  of  breast  cancer 

c.  Other? 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1.  Facts  about  breast  cancer  deaths 

na 

X 

na 

2.  Ways  to  help  lessen  the  risk  of  breast 
cancer 

na 

X 

X 

na 

3.  Other? 

na 

Because  this  question  was  not  asked  in  all  focus  groups,  the  responses  are  mixed. 
Those  persons  in  the  upper  to  middle  income  African  American  group  that 
suggested  "facts"  also  spoke  to  the  importance  of  presenting  the  information  in  a 
positive  format  and  not  heighten  fear. 
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6.  Which  topics  about  breast  cancer  would  you  listen  to?  Respondents  were  asked  to  pick 
only  one  from  the  following  list. 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1  .Breast  cancer  and  African  American 

women 

2 

0 

3 

0 

2.  Do  genetics  and  family  history  play  a 
role  in  breast  cancer? 

2 

3 

1 

0 

3.  Breast  cancer,  the  pill  and  menopause 

2 

0 

0 

2 

: 

4.  How  to  lower  your  risk  of  breast  cancer 

4 

3 

5 

0 

5.  What  is  breast  cancer? 

0 

0 

1 

0 

6.  Are  you  at  risk  of  getting  breast  cancer? 

0 

0 

1 

1 

7.  Your  lifestyle  and  how  it  effects  breast 
cancer? 

0 

0 

0 

2 

8.  How  can  breast  cancer  be  treated? 

0 

0 

0 

! 

3 

7.  Are  there  any  topics  you  might  want  to  learn  about  cancer  that  you  feel  would  get  your 
attention  if  you  heard  about  it  through  the  media? 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1.  Prevention/detection/ treatment 

X 

X 

X 

2.  Research 

X 

X 

3.  Other 

X 

X 

X 

X 

The  range  of  responses  to  this  question  varied.  The  detail  report  shows  the  greatest 
concentration  of  responses  related  to  the  category  of  prevention,  treatment  or 
research  or  to  the  category  defined  as  "other".  Marked  differences  exist  within 
each  category.  It  should  be  noted  that  one  respondent  from  the  low  to  moderate 
income  African  American  group  pointed  out  that  she  would  not  listen  at  all  to 
information  about  breast  cancer. 
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8.  Have  you  received  any  information  about  breast  cancer  in  the  last  several  years?  If  yes, 
where?  (Presented  in  yes/no  responses)  and  from  which  source  would  you  rather  learn  about 
breast  cancer?  (Represented  in  a  numerical  response) 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/ 

mod 

White 

up/mid 

1.  Advertising 

no/1 

no/2 

no/0 

no/2 

2.  News  media 

yes/0 

no/4 

no/0 

no/2 

3.  Your  doctor 

no/6 

yes/3 

no/2 

no/4 

4.  Health  organization 

yes/0 

yes/3 

no/0 

yes/0 

5.  Other 

no/0 

yes/0 

yes/1 

yes/0 

The  responses  from  the  focus  group  participants  cover  two  questions.  A  yes/no 
response  is  provided  to  the  question  about  whether  information  has  been  received 
in  the  last  several  years.  A  numerical  response  is  provided  indicating  the  number  of 
participants  having  a  preference  for  how  they  should  receive  information  on  breast 
cancer. 

9.  How  would  you  design  a  mailer?  • 


Response  Type 

African 

Amer. 

low/mod 

African 

Amer. 

up/mid 

Mixed 

low/mod 

White 

up/mid 

1.  Appearance 

bright  colors, 
postcard 

colorful, 

texture, 

oversized, 

personalized 

persona¬ 
lized,  message 
on  envelope 

Large  print, 
post¬ 
card,  easy 
reader 

2.  Focus  of  message 
content 

"Don't  just  do  it 
for  you,  do  it  for 
your  children" 

"Do  you  want 
to  take  a  chance 
with  your  life?" 

"It  can  happen 
to  you—show 
positive" 

3.  Miscellaneous 
considerations 

incentive  gift 

coupon  and 
notice  of  free 
gift 

Reference 
sources  of 
authority 

Schedule  of 

mammo¬ 
gram  site 
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A  second  series  of  two  focus  groups  that  were  identified  to  react  to  tentative  mail  designs 
and  assist  in  offering  information  and  input  on  the  mail  design  that  would  bring  the  most 
effective  response.  These  two  groups  included  the  members  of  the  first  low  to  moderate 
Afiican  American  focus  group,  and  a  low  to  moderate  white  income  group  of  women  who 
were  contacted  for  the  first  time  to  provide  their  reaction  to  the  proposed  mail  designs. 

Each  group  was  given  choices  of  four  postcard  designs  to  critique.  Input  from  the  two  focus 
groups,  health  educator  advisors,  and  project  staff  were  reviewed,  and  resulted  in  the  final 
postcard  design  (See  Appendices  B  and  C). 

The  folded  postcard  (Appendix  C)  was  mailed  to  all  participants  receiving  Treatment  A-- 
the  automated  telecommunication  system.  Subjects  in  Treatment  B--  the  live  counselor 
telecommunication  system  received  the  double-sided  postcard  (Appendix  B). 

TASK  7:  Conduct  at  least  two  comparable  tests  of  the  system. 

All  Activities  related  to  Task  7  were  successfully  completed.  Over  the  three-  year 
research  period,  one  pre-test  and  four  test  efforts  were  conducted,  as  follows: 

{ 

•  A  Pre-Test,  conducted  in  Baltimore,  Maryland,  prior  to  initiating  Test  One.  This  pre¬ 
test  examined  the  response  patterns  of  callers,  using  a  small  sampling  fi-om  the  targeted 
households.  The  pretest  was  designed  to  provide  information  to  help  determine  the 
number  of  mailings  for  Test  One  and  subsequent  tests.  In  addition,  the  pretest  was 
designed  to  obtain  any  information  regarding  the  subjects’  reactions  to  the  content  and 
the  overall  system.  This  information  was  used  to  make  adjustments  to  the  system  before 
Test  One. 

•  Test  One,  conducted  in  Baltimore,  Maryland.  This  statistically  significant  test  was  a 
comparative  assessment  of  caller  preferences  in  four  target  groups  for  an  automated 
telephone  information  system  versus  a  live  counselor  referral  service. 

•  Test  Two,  conducted  in  Annapolis,  Maryland.  This  anecdotal  assessment  compared 
respondent  patterns  to  an  automated  telephone  information  system  versus  an  on-line 
bulletin  board  system  in  an  Afiican  American  low-income  target  group. 

•  Test  Three,  conducted  in  Philadelphia,  Pennsylvania.  This  statistically  significant 
test  was  a  comparative  assessment  of  caller  preferences  in  four  target  groups  for  an 
automated  telephone  information  system  versus  a  live  counselor  referral  service.  Test 
Three  mirrored  the  design  of  Test  One  in  a  different  geographic  area,  and  was 
designed  to  validate  the  statistically  significant  results  of  Test  One. 

•  Test  Four,  conducted  in  Baltimore,  Maryland.  This  anecdotal  test  compared  caller 
response  patterns  to  different  promotional  media  and  different  media  messages  in  an 
African  American  low-income  target  group. 
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TASK  8:  Research,  define,  and  identify  appropriate  future  modes  of  sustaining  the 
information  system  beyond  the  test  phase. 

All  activities  related  to  Task  8  were  successfully  completed. 

Research  was  conducted  related  to  determining  appropriate  target  markets  for  the  Breast 
Health  Information  System,  as  well  as  the  geographic  targeting  process.  We  determined 
that  health-related  organizations  and  institutions,  insurance  companies,  and  human 
resource  departments  could  best  benefit  from  a  "health  information  line".  The  system 
design  that  was  developed  and  tested  during  the  research  period  for  breast  health 
information  could  easily  be  used  as  a  "template"  and  applied  to  any  area  of  health 
information  and  disease  prevention. 

To  maximize  the  Breast  <  Health  Information  System’s  commercial  appeal,  extensive 
research  was  conducted  in  designing  the  system’s  product  features,  menuing  structure,  and 
user  interface.  Between  Tests  Three  and  Four,  the  system  was  re-designed  based  on  user, 
focus  group,  and  evaluator  feedback  collected  over  the  first  three  tests.  The  result  was  a 
more  marketable,  user  friendly  system.  In  addition,  one  of  the  main  focus  points  of  the 
system  “re-design”  was  to  create  a  "health  information  template”  that  would  accommodate 
potential  buyers  from  different  market  segments  and  of  different  sized  organizations.  The 
resulting  product  provides  convenience  and  flexibility  in  its  design,  so  as  to  accommodate  a 
multitude  of  different  potential  buyers. 

During  the  project  period,  extensive  focus  group  sessions  were  conducted  to  both  validate 
test  results,  and  to  obtain  target  group  evaluations  of  the  system  and  reconunended 
improvements.  These  focus  group  conunents  and  suggestions  were  used  in  re-designing 
the  system's  menu  structure,  and  in  defining  our  target  market. 

Over  the  three-year  research  period,  many  meetings  were  conducted  with  project 
advisors,  private  health  industry  professionals,  and  health-related  organizations 
concerning  the  applicability  of  the  health  information  line  to  their  health  outreach  and 
disease  prevention  efforts.  Health  information  officers  at  national  health  organizations, 
such  as  the  Cancer  Information  Service,  were  contacted  and  meetings  were  held  to 
discuss  the  possibilities  of  implementing  such  a  system  at  their  locations. 

Officials  at  the  Cancer  Information  Service  were  extremely  impressed  with  the  results 
achieved  from  our  tests,  and  indicated  a  strong  interest  to  work  with  TeleSonic  on  an 
ongoing  basis.  They  also  requested  copies  of  our  test  results  to  share  with  management  at 
the  Cancer  Information  Service. 

Test  Two  (comparison  of  the  automated  system  to  an  on-line  computer  system)  was 
designed  and  implemented  with  an  eye  towards  sustaining  the  system.  Discussions  were 
held  with  both  the  physician's  office  used  for  testing,  as  well  as  management  from  the 
Annapolis  Housing  Authority,  about  the  possibilities  of  implementing  such  a  system  in 
these  low-income  communities. 


52 


A  marketing-oriented  journal  article  targeted  toward  health  information  and  outreach 
professionals  was  also  written  and  submitted  detailing  the  research  process  used,  the  system 
that  was  designed,  and  the  results  obtained. 

Additional  ideas  generated  for  sustaining  the  system  included  developing  a  "fee  for  use" 
service  where  TeleSonic  operated  the  system  for  a  client  in  exchange  for  a  fee  (as  opposed 
to  selling  an  organization  a  system).  Another  idea  included  using  paid  sponsorship  ads 
within  the  system  to  help  organizations  support  the  costs  of  operating  the  system. 

As  a  follow-on  effort  to  the  study,  TeleSonic  will  consider  sharing  approximately  10  to  15 
copies  of  the  report  results  in  a  specially  prepared  marketing  package.  The  package  will  be 
distributed  to  health  care  organizations  on  both  the  provider  side  and  the  managed  care  side. 
The  report  results  have  implications  for  health  care  providers  in  other  fields  beside  breast 
cancer.  For  example,  providers  of  information  to  arthritis  patients,  diabetes  patients,  HIV¬ 
positive  patients,  and  others  may  all  find  value  fi’om  the  results  of  TeleSonic's  study,  which 
shows  many  effective  ways  to  share  health  information  to  informationally-hard-to-reach 
populations. 

TASK  9:  Document  and  report  results. 

a)  Implement  documentation  procedures 

b)  Produce  internal  quarterly  reports 

c)  Produce  annual  and  final  reports 

d)  Provide  ongoing  program  administrative  support. 

All  activities  related  to  Task  9  were  successfully  completed.  During  the  research  period, 
documentation  procedures  were  designed  and  implemented,  an  administrative  manual 
was  developed,  internal  reports  were  generated,  two  armual  reports  and  a  final  report  were 
developed,  and  ongoing  program  administrative  support  was  provided. 
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VI.  Conclusions 


A.  SUMMARY  OF  IMPLICATIONS  OF  THE  COMLETED  RESEARCH 


Anticipated  Results  versus  Results  Achieved 

TeleSonic's  originally  anticipated  results  of  the  research  study  and  development  project 
included: 

1.  The  successful  design,  development,  and  operation  of  a  multimedia 
automated  telephone  information  system  which  includes  an  on-line 
interactive  caller  quiz,  an  audio  library  of  static  and  dynamic  (changing) 
information  messages,  transfer  ability  to  a  live  helpline  assistant,  caller 
message  taking  and  feedback  capability,  and  supplementary  print 
material  available  immediately  by  fax,  or  within  a  few  days  by  mail. 

This  anticipated  result  was  successfully  achieved.  A  multimedia  automated 
telephone  information  system  was  developed  that  included  an  on-line  interactive 
caller  quiz,  an  audio  library  of  static  and  dynamic  (changing)  information  messages, 
transfer  ability  to  a  live  helpline  assistant,  caller  message  taking  and  feedback 
capability,  and  supplementary  print  material  available  immediately  by  fax,  or  within 
a  few  days  by  mail. 


2.  An  increase  in  the  number  of  minority  and  underserved  females  seeking 
breast  cancer  information. 

In  both  Tests  One  and  Three,  statistically  significant  preferences  were  shown  by  low- 
moderate  income  level  women  for  the  automated  telephone  system.  This  statistical 
preference  indicates  that  the  automated  system  is  an  effective  way  of  increasing  the  number 
of  minority  and  underserved  women  to  obtain  information  about  breast  cancer. 


3.  An  increase  in  the  number  of  females  who  take  proactive  steps  to  sign 
up  for  mammography  screening  and/or  make  an  appointment  with  a 
physician. 

In  Tests  One  and  Three  this  information  was  difficult  to  track,  and  information  obtained  was 
based  on  self-reports  only  (through  follow-up  surveys).  To  enable  us  to  get  better 
information  in  this  area,  we  re-designed  the  automated  system  for  Test  Four  to  allow  us  to 
more  directly  track  the  number  of  women  who  signed  up  for  a  mammography  screening  or 
who  made  an  appointment  with  a  physician. 
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In  Test  Four  the  following  results  were  obtained: 

□  Of  the  94  library  messages  listened  to,  1 1  related  to  mammograms. 

□  Of  the  41  prevention  step  messages  listened  to,  22  related  to  mammograms,  7  were 
about  breast  self-examination,  5  were  about  doctor  appointments,  and  8  were  about  risk 
factors. 

□  Of  the  five  transfers  to  Johns  Hopkins  Breast  Clinic,  all  five  were  to  make  an 
appointment  for  a  mammogram. 

□  Of  the  nine  documents  requested,  six  were  for  information  about  mammograms  and 
three  were  for  information  about  breast  self-examination. 

□  Of  the  31  survey  participants,  9  indicated  they  would  do  a  monthly  breast  self- 
examination,  14  stated  they  would  get  a  mammogram,  and  5  said  they  would  make  an 
appointment  with  a  doctor. 

These  results  show  a  clear  preference  by  women  for  information  about  mammograms  and 

low-cost  appointment  options.  Women  were  much  more  eager  and  willing  to  get  a 

mammogram  than  to  make  a  doctor's  appointment. 


4.  The  collection  of  previously  unavailable  pertinent  data  measuring  the 
efficacy  of  live  telephone  health  information  services  versus  automated 
telephone  health  information  systems. 

In  both  Test  One  and  Test  Three,  low-moderate  income  level  callers  showed  an  overall 
preference  for  the  automated  telephone  information  system  over  a  live  counselor  referral 
system.  As  detailed  earlier  in  this  report,  multiple  focus  group  sessions  conducted  during 
the  research  period  validated  these  findings. 


5.  The  enhancement  of  the  breast  cancer  information  and  communication 
infrastructure  for  use  by  physicians,  patients,  and  concerned  family 
members  and  friends. 

The  system  designed  during  the  research  period  and  the  information  developed  represent  a 
unique  and  new  way  for  physicians,  patients,  family  members,  and  fnends  to  get  easy 
information  about  breast  cancer  and  breast  cancer  prevention. 

Comparison  of  Hypotheses  to. Results  Obtained 

This  section  compares  our  hypothesis  and  related  hypotheses  to  the  results  obtained 
during  the  research  period. 

Major  Hypothesis:  Multimedia  automated  telephone  systems  enhance  live  referral 
alone  as  a  vehicle  to  reach  the  informationally  hard-to-reach. 
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VI.  Conclusions 


A.  SUMMARY  OF  IMPLICATIONS  OF  THE  COMLETED  RESEARCH 


Anticipated  Results  versus  Results  Achieved 

TeleSonic's  originally  anticipated  results  of  the  research  study  and  development  project 
included: 

1.  The  successftil  design,  development,  and  operation  of  a  multimedia 
automated  telephone  information  system  which  includes  an  on-line 
interactive  caller  quiz,  an  audio  library  of  static  and  dynamic  (changing) 
information  messages,  transfer  ability  to  a  live  helpline  assistant,  caller 
message  taking  and  feedback  capability,  and  supplementary  print 
material  available  immediately  by  fax,  or  within  a  few  days  by  mail. 

This  anticipated  result  was  successfully  achieved.  A  multimedia  automated 
telephone  information  system  was  developed  that  included  an  on-line  interactive 
caller  quiz,  an  audio  library  of  static  and  dynamic  (changing)  information  messages, 
transfer  ability  to  a  live  helpline  assistant,  caller  message  taking  and  feedback 
capability,  and  supplementary  print  material  available  immediately  by  fax,  or  vrithin 
a  few  days  by  mail. 


2.  An  increase  in  the  number  of  minority  and  underserved  females  seeking 

breast  cancer  information. 

In  both  Tests  One  and  Three,  statistically  significant  preferences  were  shown  by  low- 
moderate  income  level  women  for  the  automated  telephone  system.  This  statistical 
preference  indicates  that  the  automated  system  is  an  effective  way  of  increasing  the  number 
of  minority  and  underserved  women  to  obtain  information  about  breast  cancer. 


3.  An  increase  in  the  number  of  females  who  take  proactive  steps  to  sign 
up  for  mammography  screening  and/or  make  an  appointment  with  a 
physician. 

In  Tests  One  and  Three  this  information  was  difficult  to  track,  and  information  obtained  was 
based  on  self-reports  only  (through  follow-up  surveys).  To  enable  us  to  get  better 
information  in  this  area,  we  re-designed  the  automated  system  for  Test  Four  to  allow  us  to 
more  directly  track  the  number  of  women  who  signed  up  for  a  mammography  screening  or 
who  made  an  appointment  with  a  physician. 
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Over  the  three-year  research  period,  TeleSonic  found  that  when  faced  with  the  choice  of 
getting  preventive  health  care  information  from  an  automated  telephone  information 
system  versus  a  live  counselor,  both  African  American  and  White  lower  income  women 
consistently  choose  the  automated  telephone  information  system.  In  contrast,  middle-to- 
upper  income  White  women  choose  to  speak  to  a  live  counselor.  These  results  support 
the  project's  major  hypothesis:  that  using  a  multimedia  automated  telephone  information 
system  is  an  effective  means  of  reaching  "informationally  hard-to-reach"  populations,  and 
getting  them  to  take  proactive,  preventive  care  steps  against  breast  cancer.  Based  on 
these  results,  we  can  conclude  that  TeleSonic's  method  has  considerable  implications  for 
many  areas  of  health  care  marketing,  prevention,  and  treatment,  particularly  in  targeting 
and  reaching  traditionally  "hard-to-reach"  populations. 


Related  Hypotheses: 

1.  Callers  overall  will  show  a  greater  tendency  to  call  a  multimedia  automated 
telephone  system  combined  with  live  referral  than  to  call  the  live  referral 
alone. 

In  both  Test  One  and  Test  Three,  low-moderate  income  level  callers  showed  an  overall 
preference  for  the  automated  telephone  information  system  (combined  with  a  live 
counselor  transfer  option)  over  a  live  counselor  referral  system. 

As  detailed  earlier  in  this  report,  multiple  focus  group  sessions  conducted  during  the 
research  period  validated  these  findings. 


2.  Within  racial  groups,  callers  at  different  socio-economic  status  (SES)  levels 
will  show  significant  differences  in  caller  preferences  for  use  of  a  multimedia 
automated  telephone  system  with  a  live  referral  option  compared  to  live 
referral  alone. 

As  stated  above,  over  the  three-year  research  period  TeleSonic  found  that  when  faced 
with  the  choice  of  getting  preventive  health  care  information  from  an  automated 
telephone  information  system  versus  a  live  counselor,  both  Afiican  American  and  White 
lower  income  women  consistently  choose  the  automated  telephone  information  system. 
In  contrast,  middle-to-upp?r  income  White  women  choose  to  speak  to  a  live  coimselor. 

Also,  as  detailed  earlier  in  this  report,  multiple  focus  groups  conducted  by  racial  and 
income  groups  confirmed  these  findings. 


3.  Callers  at  lower  SES  levels  will  show  a  significant  preference  for  the  use  of  a 
multimedia  automated  telephone  system  with  a  live  referral  option  compared 
to  live  referral  alone. 
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As  stated  above,  over  the  three-year  research  period  TeleSonic  found  that  when  faced 
with  the  choice  of  getting  preventive  health  care  information  from  an  automated 
telephone  information  system  versus  a  live  counselor,  both  African  American  and  White 
lower  income  women  consistently  choose  the  automated  telephone  information  system. 
In  contrast,  middle-to-upper  income  White  women  choose  to  speak  to  a  live  counselor. 

Also,  as  detailed  earlier  in  this  report,  multiple  focus  groups  conducted  by  racial  and 
income  groups  confirmed  these  findings. 


4.  Callers  with  higher  SES  levels  will  show  significant  differences  by  racial 
groups  in  their  preference  for  the  use  of  a  multimedia  automated  telephone 
system  with  a  live  referral  option  compared  to  live  referral  alone. 

As  stated  above,  over  the  three-year  research  period  TeleSonic  found  that  when  faced 
with  the  choice  of  getting  preventive  health  care  information  from  an  automated 
telephone  information  system  versus  a  live  counselor,  both  African  American  and  White 
lower  income  women  consistently  choose  the  automated  telephone  information  system. 
In  contrast,  middle-to-upper  income  White  women  choose  to  speak  to  a  live  counselor. 

Also,  as  detailed  earlier  in  this  report,  multiple  focus  groups  conducted  by  racial  and 
income  groups  confirmed  these  findings. 


5.  Callers  with  lower  SES  across  racial  groups  will  show  a  preference  for  the  use 
of  a  multimedia  automated  telephone  system  with  a  live  referral  option 
compared  to  live  referral  alone. 

In  both  Test  One  and  Test  Three,  callers  showed  an  overall  preference  for  the  automated 
telephone  information  system  (combined  with  a  live  counselor  transfer  option)  over  a  live 
counselor  referral  system. 

; 

As  detailed  earlier  in  this  report,  multiple  focus  group  sessions  conducted  during  the 
research  period  validated  these  findings. 


6.  A  culturally  sensitive  outreach  strategy  will  increase  caller  volume  compared 
to  a  less  culturally  focused  outreach  strategy. 

In  Test  Four,  TeleSonic  compared  caller  responses  to  a  generic  versus  a  culturally  sensitive 
outreach  strategy  across  three  different  types  of  media:  direct  mail,  supermarket  flyers,  and 
radio  public  service  announcements.  Results  from  this  test  varied  by  promotional  method, 
and  were  thereby  inconclusive.  In  the  case  of  direct  mail,  callers  preferred  the  "treated" 
(culturally  sensitive)  piece.  In  the  supermarket  test,  more  callers  preferred  the  "untreated" 
piece;  however,  as  stated  earlier  in  this  report,  TeleSonic  believes  that  this  may  be  related 
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to  the  uneven  distribution  of  the  treated  versus  untreated  pieces  at  the  supermarkets.  In  the 
radio  test,  callers  preferred  the  untreated  message. 


7.  Callers  in  different  geographical  areas  will  show  a  significant  preference  for  a 
multimedia  automated  telephone  system  with  live  referral  compared  to  live 
referral  alone. 

Test  One  and  Test  Three  were  designed  to  test  the  same  test  design  in  two  different 
geographical  areas  in  an  effort  to  "validate"  results.  In  both  tests/geographical  areas, 
callers  showed  an  overall  preference  for  the  automated  system. 


Implications  of  the  Completed  Research 

Results  of  TeleSonic's  study  have  considerable  implications  for  many  areas  of  health  care 
marketing,  prevention,  and  treatment,  particularly  in  targeting  and  reaching  large  groups 
of  traditionally  "hard-to-reach"  populations. 

Because  TeleSonic's  three-year  test  has  shown  that  the  method  is  highly  effective  for 
reaching  specific  populations  and  then  encouraging  them  to  take  preventive  care  actions, 
any  company  or  organization  that  is  interested  in  pinpointing  specific  populations,  as  well 
as  any  group  involved  in  prevention  and  outreach  can  gain  substantial  target  marketing 
benefits  from  this  method. 

Insurance  companies  and  health  care  organizations  looking  for  ways  to  control  costs  can 
use  this  method  to  target  high-risk  individuals  and  provide  them  with  preventive  care 
information.  State  and  County  Health  Departments  can  identify  high  incidence  areas  of 
any  disease  to  cost-effectively  target  outreach  efforts.  Hospitals,  research  centers,  and 
national  health  organizations  and  associations  can  target  specific  populations  to  refine 
their  health  promotion  and  outreach  efforts.  In  addition,  all  these  groups  can  expand  and 
enhance  targeted  outreach  efforts  by  offering  a  24-hour  telephone  information  service 
where  callers  can  get  sensitive  health  care  information  in  a  private,  non-threatening,  and 
convenient  way. 

The  result  is  an  innovative  technique  that  sets  new  standards  in  target  marketing  and 
opens  new  doors  in  the  areas  of  disease  prevention  and  outreach. 


B.  RECOMMENDED  CHANGES  AND  FUTURE  WORK 

Based  on  research  conducted  under  Task  8  of  this  project,  we  determined  that  health- 
related  organizations  and  institutions,  insurance  companies,  and  human  resource 
departments  could  best  benefit  from  a  "health  information  line".  The  system  design  that 
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was  developed  and  tested  during  the  research  period  for  breast  health  information  could 
easily  be  used  as  a  "template"  and  applied  to  any  area  of  health  information  and  disease 
prevention. 

To  maximize  the  Breast’ Health  Information  System’s  commercial  appeal,  extensive 
research  was  conducted  in  designing  the  system’s  product  features,  menuing  structure,  and 
user  interface.  In  addition,  many  focus  group  sessions  were  conducted  to  get  the  user's 
perspective.  The  resulting  product  provides  convenience  and  flexibility  in  its  design,  so  as 
to  accommodate  a  multitude  of  different  potential  buyers. 

TeleSonic's  future  work  will  center  on  marketing  the  breast  health  information  system,  as 
well  as  the  "health  information  template"  to  the  target  market  described  above. 

One  area  where  we  recommend  that  future  work  could  be  highly  beneficial  is  in  the  area 
of  further  refined  income  level  assessments.  The  current  research  assessed  the  behavior 
patterns  of  low-moderate  ($0  to  $24,999  annual  household  income)  versus  middle-upper 
($25,000  and  above  annual  household  income)  income  level  persons.  Based  on  the  results 
achieved  and  the  statistically  significant  preference  of  low-moderate  income  groups  for 
the  automated  system,  we  believe  that  a  further  refinement  of  income  level  assessment  to 
coincide  with  census  breakdowns  would  be  highly  beneficial.  Great  insight  into  the 
behavior  patterns  of  lower  income  ($0  to  $14,999  annual  household  income)  versus  low- 
moderate  income  persons  ($15,000  to  $24,999  annual  household  income)  could  be  gained 
through  this  analysis.  Based  on  focus  group  discussions  and  market  research,  TeleSonic 
suspects  there  may  exist  great  differences  in  these  behavior  patterns,  and  thus  better  ways 
of  reaching  these  persons. 

Another  area  for  potential  further  research  includes  developing  and  testing  a  more 
elaborate  tracking  structure  for  determining  the  subsequent  actions  taken  by  users  of  the 
automated  system.  This  type  of  research  would  provide  very  powerful  data  on  the 
effectiveness  of  automated  telephone  technology. 

One  other  area  for  future  research  includes  development  and  testing  of  a  breast  health 
information  system  that  utilizes  touchscreen  technology.  Through  both  the  focus  groups 
as  well  as  Test  two  (the-on-line  computer  test),  we  learned  that  literacy  is  a  big  roadblock 
for  many  lower  income  persons  in  obtaining  health-related  information.  A  touchscreen 
system  placed  in  public  areas  or  health  facilities  could  give  persons  with  literacy 
difficulties  access  to  the  latest  information  about  breast  cancer. 
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